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1 Introduction 


Since its origins in the early 1990s, the charter school sector has grown to reach, as of 2016, 
more than 6,900 schools serving about 3.1 million children (National Alliance for Public Char- 
ter Schools, 2016). This rapid expansion has given rise to a large, rich literature on charter 
school effectiveness (Abdulkadiroglu et al., 2011; Dobbie and Fryer, 2011; Abdulkadiroglu et 
al., 2016).! These publicly funded schools were initially conceived as a means to spur innova- 
tion in and competition for surrounding traditional public schools, yet growing concerns have 
emerged about charter schools’ potential negative impact on non-charter students. Such con- 
cerns have had real-world effects in Massachusetts, where in November 2016 voters rejected 
a ballot initiative that would have added up to 12 new charter schools per year. This paper 
investigates the fiscal and educational impact of charter expansion on traditional public schools 
by exploiting a 2011 reform that raised the cap on charter schools in Massachusetts. 

Causal evidence on the fiscal impact of charter schools is very limited (Epple et al., 2015), 
yet this question is central to charter school policy debates. Charter school proponents claim 
that increased competition generated by charter expansion might induce districts to strategi- 
cally reallocate spending (Hoxby, 2003).? Opponents, meanwhile, argue that charter schools 
drain resources from traditional public schools. In most states, when a student switches from a 
traditional public school to a charter school, the funding follows the student. This might force 
school districts facing fixed costs to cut per-pupil spending on variable outlays, hampering 
their ability to respond to competition. To counteract this, however, several states, including 
Massachusetts, have refund schemes that temporarily compensate school districts for lost rev- 
enues due to charter expansion. Taking all these forces together, the net fiscal impact of charter 
expansion is unclear (Arsen and Ni, 2012; Bifulco and Reback, 2014). 

The first objective of this paper is to isolate exogenous changes in the share of students 
who enroll in charter schools to assess the causal fiscal impact of charter school expansion 
on sending districts. By fiscal impact, we mean how charter expansion impacts (i) districts’ 
average per-pupil expenditures on non-charter students, (ii) the share of these expenditures 
devoted to fixed and variable costs, and (iii) the share devoted to support services, instruction, 
and salaries. Understanding the potential fiscal impact of charter expansion is fundamental 


'See also Epple et al. (2015) for an excellent literature review on charter schools. 

°A large literature has looked at the competition effect of both private and public schools (see Epple et al. 
(2017) for a recent literature review, also Card et al. (2010) and Clark (2009). Yet charter schools differ from 
both. Unlike private schools, charters do not charge tuition and are publicly funded. In that respect, charter 
schools compete with traditional public schools for funds, while private schools do not. The competition effect of 
charter schools also differs from that of traditional public schools. First, charter schools’ opening and expansion 
are regulated differently. As a result, charter schools have considerably expanded in the last 30 years, while the 
number of traditional public schools has remained relatively constant. Secondly, in Massachusetts, traditional 
public schools receive temporary financial aid when students switch to a charter school but not when students 
switch to a different public school or district. 

Bifulco and Reback (2014) provide case studies of traditional public schools’ financial adaptations to declin- 
ing enrollment in Albany and Buffalo, New York. Arsen and Ni (2012) find that higher charter school enrollment 
levels in Michigan school districts are strongly associated with declining fund balances. 


for several reasons. First, per-pupil expenditure can be a determinant of student achievement 
(Jackson et al., 2016). Second, spending on fixed costs (typically building maintenance or debt 
interest) or support services might not generate student progress to the same extent as spending 
on instruction, textbooks, or teacher salaries, which are more variable costs. 

A second focal point in the debate on charter expansion is its spillover effects on non- 
charter student achievement (Cordes, 2017; Imberman, 2011; Booker et al., 2008). If charter 
schools drain resources and high-achieving peers from non-charter schools, this could harm 
student achievement in traditional public schools. Such a general equilibrium effect could 
bias upwardly estimates of charter school effectiveness. The second objective of this paper 
is therefore to revisit the question of charter expansion’s impact on student achievement by 
using a novel identification strategy. As highlighted by Epple et al. (2015) in their review of 
the charter schools literature, studies of charter expansion’s spillover effects face a number of 
methodological challenges that have rarely all been addressed in a single paper. Our study aims 
to fill that methodological gap. 

A key challenge in isolating the impact of charter expansion is the non-random initial lo- 
cation and expansion of charter schools (Glomm et al., 2005; Bifulco and Buerger, 2015).4 To 
deal with this endogeneity, we exploit a Massachusetts reform that led to charter sector expan- 
sion. In 2011, the state raised the limit on district funding allocable to charter schools from 9% 
to 18% in districts with student performance in the lowest 10% of districts statewide.” In the 
four years following the reform, the share of students attending charter schools jumped from 
7% to 12% in the districts that expanded their charter sector (hereafter termed expanding dis- 
tricts). The charter share remained relatively constant at about 3% in all other districts (hereafter 
termed nonexpanding districts). We use the charter sector’s differential growth in expanding 
and nonexpanding districts as an instrument for district charter share. A potential concern with 
this is that expanding districts are non-randomly selected. The 2011 reform induced some, but 
not all, eligible districts to expand their charter sector. 

To account for district selection into expansion, we start by building a synthetic control 
for the expanding districts (Abadie and Gardeazabal, 2003; Abadie et al., 2010, 2015).° This 
method creates a data-driven weighted average of a small number of control districts that 
closely matches the expanding districts’ outcome path in the years prior to the reform. One 
of our methodological contributions is to use the comparison between expanding districts and 
the synthetic control districts as an instrument for charter share. To do so, we show how to 
ensure that expanding and synthetic control districts have similar pre-reform trends in both ex- 


4Charter schools tend to locate in districts where the population is diverse, per-pupil expenditure is high, 
teacher costs are low, and public school achievement is relatively low (Glomm et al., 2005; Bifulco and Buerger, 
2015). 

In addition, to ensure that only high-performing charter schools would open or extend, expansions in districts 
close to the 9% cap were limited to “proven providers”, that is, existing charter schools or boards of governors 
with track records of high performance. Therefore, only a subgroup of low-performing districts, in which some 
proven providers submitted applications that were accepted, actually saw expansion after the reform. 

®An alternative would have been to use districts’ pre-determined eligibility for charter expansion as an instru- 
ment for the endogenous expansion. We ruled out this strategy, however, because the take-up was too low. 


penditures and charter share.’ Though the graphical analysis of the synthetic control method 
is compelling, this method has one important drawback: the small number of synthetic control 
districts, and the fact that only post-reform years are used to measure treatment effects, makes 
statistical inference difficult. 

To address these limitations, we complement the synthetic control IV (IV-SC) approach 
with a differences-in-differences instrumental variables (IV-DiD) method. The instrumental 
variable is the interaction between the post-reform years and whether or not a district expands 
its charter segment. For nonexpanding districts, we start by re-using the group of districts 
identified and validated by the synthetic control method. We progressively enlarge this group 
to check the robustness of the results. In addition, combining IV-SC and IV-DiD gives us the 
opportunity to check that both methods yield similar results despite being based on different 
pre-trend assumptions. For DiD to be a viable instrument for charter share, the interaction 
between expanding district and post-reform years should be independent of potential outcomes, 
and this interaction should only affect student outcomes through its effects on the probability 
of charter enrollment. 

Our results reveal that higher charter attendance both increases per-pupil expenditures and 
shifts districts’ expenditures towards instruction and away from support services. The IV-SC 
method shows that, after the reform, total per-pupil expenditures increased by 4.8 % more in ex- 
panding districts than in nonexpanding districts. In addition, districts reallocate their resources. 
Per-pupil expenditures on instruction and salaries increased more in expanding districts than in 
nonexpanding districts by 7.5% and 5.2%, respectively. On the other hand, per-pupil expen- 
ditures on support services drop by 4.4% more in expanding districts.* The IV-DiD estimates 
confirm these results and provide additional evidence for the competition and fixed costs mech- 
anisms. 

We then investigate the impact of charter expansion on student achievement. This raises ad- 
ditional challenges in terms of identification. Ample evidence shows that charter students differ 
from non-charter students and that this selection changes when charter schools expand (Epple 
et al., 2015; Baude et al., 2014; Cohodes et al., 2016). Restricting achievement outcomes to 
non-charter students, as we do for expenditures outcomes, would bias estimations. Unlike most 
previous studies, we therefore estimate charter expansion’s causal impact on the achievement 
of all students while controlling for individual charter enrollment (with time-varying effects).’ 
Finally, a common concern is that charter schools locate in areas that have experienced increas- 
ing or decreasing trends in achievement (Imberman, 2011). We combine an IV-DiD, which 
assumes parallel pre-trends in outcomes, and an IV-SC, which imposes them, as an important 


In other words, similar pre-reform trends are needed for both the first stage and the reduced form. 

8The synthetic control method relies on a number of choices made regarding the outcome predictor variables, 
the donor pool, and the method used to compute the predictor variable weights. We test the robustness of our 
results by showing results for six different specifications. 

°Controlling for individual charter enrollment requires us to account for student selection into charter schools. 
We use charter admissions lottery offers as an instrumental variable for individual charter enrollment (Angrist et 
al., 2010; Dobbie and Fryer, 2011, 2016; Abdulkadiroglu et al., 2011; Angrist et al., 2013, 2016; Cohodes et al., 
2016). 


robustness check. 

Our results show that charter sector expansion has a positive impact on student achievement, 
although the effects are not always significant. In both math and English language arts (ELA), 
the IV-SC method indicates a small but not significant improvement in student achievement. 
The IV-DiD estimates reveal that charter expansion makes a positive and significant impact on 
student achievement. Our estimates suggest that moving from 10% to 15% of students attend- 
ing charter schools would increase the achievement of non-charter students by 0.03 standard 
deviations in math and by 0.02 in ELA. These results confirm, to some extent, the findings of 
previous studies showing charter expansion has a limited impact on student achievement (Bet- 
tinger, 2005; Imberman, 2011; Zimmer and Buddin, 2009; Davis, 2013; Sass, 2006; Winters, 
2012). 

Our results stand in contrast to prior studies that find a negative fiscal impact of charter 
expansion on traditional school districts (Arsen and Ni, 2012; Bifulco and Reback, 2014; Ladd 
and Singleton, 2018; Cook, 2018). This may be because we provide the first causal evidence 
in a state that temporarily compensates districts for the revenue they lose when students move 
to a charter school.'° The effect of charter expansion in such contexts is highly policy relevant, 
as several states, among them Illinois, New Hampshire, New York, and Pennsylvania, have 
adopted temporary refund schemes. However, with a temporary refund scheme, we might also 
expect the long- and short-run effects of charter expansion to differ. What happens to district 
expenditures and student achievement after the refund ends? 

To look at the longer-run, post-refund consequences of charter expansion, we analyze char- 
ter school openings prior to 2011. We use an event-study analysis of all charter school openings 
that persistently raised the charter share. Reassuringly, this method largely replicates our pre- 
vious results over the three- to four-year timeline. Our event study suggests that the effects 
of charter expansion on both expenditures and achievement vanish in the longer run, and in 
particular after the refund ends. Interestingly, the positive effects on achievement are largest 
five to six years after opening, which fits with previous findings that it takes several years for 
increased spending to impact achievement (Jackson et al., 2016; Lafortune et al., 2018). These 
results indicate that the refund scheme may be effectively insulating districts from the short-run 
financial shock due to expansion so that they can adjust in the long run. 

In addition to contributing to the long-running debate on the consequences of charter expan- 
sion for school districts, our empirical results are of immediate policy interest. In November 
2016, a ballot question on whether or not to expand the charter school sector in Massachusetts 
drew national attention (The New York Times, 2016). A majority of the state voted against the 
charter cap expansion in what is perceived nationally as a landmark decision. Given that about 
half of American states regulate charter expansion by setting caps, we expect discussions on 
the benefits and costs of raising these caps to become more frequent in the future. This analysis 


'OBifulco and Reback (2014) present case studies of traditional public schools adapting to enrollment declines 
in Albany and Buffalo, New York, where a temporary aid program exists. They do not provide causal estimates, 
however. 


will bring some evidence into that debate. 


2 Background 


The Massachusetts Charter School Sector 


The first Massachusetts charter schools opened in 1995, following the 1993 Massachusetts Ed- 
ucation Reform Act, which allowed non-profit organizations, teachers, or other groups wishing 
to operate charter schools in Massachusetts to submit applications to the state’s Board of Ed- 
ucation. There are no for-profit charter schools in Massachusetts. Once authorized, charter 
schools in Massachusetts share a number of organizational features with charter schools in 
other states. Typically, charter schools are free to organize instruction around a philosophy 
or curricular theme and to adopt a longer school day and year than traditional public schools. 
Charter schools also have more discretion over staffing and compensation than traditional pub- 
lic schools. Most of the time, charter schools are exempt from local collective bargaining 
agreements. 

Massachusetts charter schools face stringent state accountability requirements. All charter 
schools operate under five-year charters granted to an independent board of trustees. Charter 
schools are therefore required to file for renewal every five years and are held accountable annu- 
ally via reports, financial audits, and site visits. Renewal applications must show that the school 
has been successful in terms of student achievement. As a result of this strict renewal process, 
since 1994, 29 approved charter schools have either never opened, closed, or surrendered their 
charters. 


Charter school expansion 


Since its origins in the mid 1990s, the charter school sector in Massachusetts has grown to 
80 schools serving more than 40,000 children in the 2016-2017 school year. Charter students 
represent about 4.2% of the PK-12 public school population (Massachusetts Department of 
Elementary and Secondary Education, 2017). 

This expansion has been facilitated by amendments to the numerical and net school funding 
caps set forth by the Massachusetts Legislature.'! In 1997, the state adopted a 6% limit on 
district funding allocable to Commonwealth charter school tuition. That cap was raised to 9% 
in 2000.'* In 2010, a legislative amendment to the charter school statute established the current 


"Tike several other states in the U.S., Massachusetts regulates charter expansion by a system 
of caps. At the time of writing, 24 states have caps on the number of charter schools. Source: 
http://www.publiccharters.org/wp-content/uploads/2017/03/MODEL-—Report_ 
FINAL. pdf?x87663. 

Tn 1997, the numerical cap was raised to 50. The funding cap only applies to Commonwealth charter schools, 
which represent the vast majority of the charter sector. In 2016-2017, 71 of the 80 operating charter schools were 
Commonwealth. 


funding cap provisions for charter schools. In districts with student performance in the lowest 
10% of districts statewide, the 9% limit on district funding allocable to Commonwealth charter 
school was increased such that it would reach 18% by 2017. For all other districts, the 9% limit 
was unchanged. Districts are in the lowest 10% of achievement if their average math and ELA 
scores on the Massachusetts Comprehensive Assessment System have been in the lowest 10% 
statewide for two consecutive years. 

To ensure that only high-performing charter schools would open or expand, the state lim- 
ited expansions in districts close to the 9% cap to proven providers, i.e., existing operators with 
strong track records.'? Criteria for proven provider classification include performance on state 
achievement tests, enrollment, attendance, retention, attrition rates, graduation rates, dropout 
rates, suspension numbers, effective governance, and competent financial management.'* With 
applications limited to proven providers and the rigorous Massachusetts Board of Elementary 
and Secondary Education approval process, only a subgroup of low-performing districts ex- 
panded their charter sectors after the reform. 

The 9% cap was also not equally binding for all districts before the reform. At the state 
level, there was high excess demand for charter schools in 2010. At that time, almost as 
many students were on charter school waiting lists (26,708) as were actually enrolled in char- 
ter schools (28,422). Boston was the most seat-constrained district, as the Board of Education 
stopped accepting proposals for new Boston charters after expenditure reached the cap in 2008. 
Many districts in the lowest 10th percentile of student achievement were, however, far from the 
9% cap. 

The 2011 cap reform led to a significant increase in charter enrollment. Figure 1 shows 
that, in the four years following the 2011 cap increase, the proportion of students attending 
a charter school jumped from 7% to 12% in the low-performing districts that expanded their 
charter sectors after the reform (expanding districts). The charter share remained relatively 
constant at about 3% in all other districts (nonexpanding districts).!° 

Figure 2 reports the charter share evolution for middle schools only. Charter enrollment 
grew at the elementary and high school levels, though not as dramatically as in middle schools, 
where the proportion of students attending a charter school jumped from 10% to 15% in ex- 
panding districts. For that reason, we focus on middle school students when studying charter 
expansion effect on student achievement. However, we analyze fiscal spillovers for all levels 
— that is, primary, middle, and high schools — as the expenditure variables are not decomposed 
by level. 


3More specifically, proven provider status was required for charter applications in districts with the lowest 10% 
of student performance whenever additional charter enrollment would cause tuition payments to exceed 9% of the 
district’s net school spending. 

4For a complete definition of proven providers, see Massachusetts Education Laws and Regulations (603 CMR 
1.00) related to Charter schools, section 4: http: //www.doe.mass.edu/lawsregs/603cmrl1.html? 
section=04. 

Not surprisingly, Boston is one of the districts that used the raised cap to expand its charter sector. The 
percentage of students enrolled in a charter school went from 9% to 15% between academic years 2010-2011 and 
2013-2014. 


Districts Expenditures on Charter Schools and Traditional Public Schools 


The vast majority of charter school funding—about 90 percent—comes from tuition payments 
paid by the sending district, which is the district where a student resides. Policy debates often 
mention fiscal impacts of charter expansion on districts. When a student switches from a tra- 
ditional public school to a charter school, the budget follows the student. Charter schools are 
therefore criticized for draining students and resources from traditional public schools. 

Sending districts calculate their total charter school tuition payment by multiplying the 
number of students attending a charter school by a per charter student tuition amount. In 
practice, tuition amounts are roughly equal to average per-pupil spending in the sending district. 
An important feature of charter school funding in Massachusetts is the availability of state 
programs that offset the charter tuition paid by sending districts. The state funds a charter 
reimbursement program, called “Chapter 46 Aid”, that pays a portion of district tuition costs in 
the six years after an increase in the number of students attending charter schools. Specifically, 
when tuition payments increase for a school district over the prior year, the state reimburses 
that district for 100 percent of the increased cost in the first year. The state then reimburses 25 
percent of this first-year amount for each of the subsequent five years. Reflecting this six-year 
reimbursement schedule, Chapter 46 Aid is sometimes referred to as the “100/25/25/25/25/25” 
formula.'!© Appendix Table A.1 presents an example that describes the timing of Chapter 46 
Aid. The second component of Charter 46 Aid is a refund for first-year pupils entering public 
charter schools from private or home-schooled settings. To help defray this additional cost, the 
state fully reimburses this first year’s tuition. In later years, financial responsibility for these 
charter students shifts back to the district. Finally, the state fully reimburses the “facilities aid” 
that sending districts pay to charter schools to help pay for school buildings. In the 2016-2017 
school year, the total state aid amounted to $80 million, decomposed into $39.6 million for the 
100/25/25/25/25/25 formula, $34.4 million for the facilities aid, and $6.9 million for former 
private or home-schooled students."” 

These state aid programs mean that for several years after a student switches from a tra- 
ditional public school to a charter school, the district is reimbursed for at least some of the 
(average) amount it would have spent on that student. As a result, all else equal, the district’s 
revenue per pupil enrolled will increase when students transition to a charter school. How- 
ever, whether this translates into higher per-pupil expenditures is not clear. Evidence of low 
government spending elasticities and flypaper effects suggest that state aid programs might not 
translate into higher per-pupil expenditures (Feldstein, 1975; Hines and Thaler, 1995; Inman, 


'6Ror details on the tuition formula, see Massachusetts Education Laws and Regulations (603 CMR 1.00) related 
to Charter schools, section 7: http://www.doe.mass.edu/lawsregs/603cmrl1.html?section= 
07. 

Tt should be noted that in recent years the Massachusetts Legislature has not appropriated sufficient funding 
to provide sending districts with 100 percent of the reimbursements they should have received. In 2013 and 2014, 
districts received 96% and 97% of the total reimbursement. This rate dropped to 69%, 63%, and 58% in the years 
2015, 2016, and 2017. 2015 is the most recent year we consider in this anaysis, so insufficient funding will only 
impact the last year of our sample. 


2008; Fisher and Papke, 2000; Gordon, 2004; Dee and Levine, 2004). Whether charter expan- 
sion increases districts’ expenditures on non-charter students is therefore an open question. 

Charter expansion might impact not only how much revenue districts devote to traditional 
public schools but also how they spend such revenue. Here, two mechanisms might compensate 
each other. On one hand, if traditional public schools lose students, their per-pupil expenditures 
on fixed costs would mechanically go up, while per-pupil expenditures on variable costs might 
go down. Typical fixed costs are building maintenance or debt interest, while spending on 
textbooks, instruction, or teachers is usually considered to be more variable. On the other hand, 
increased competition generated by charter expansion might induce districts to shift resources 
toward inputs that are perceived as more productive in terms of student achievement (Hoxby, 
2003). Typically, districts might increase spending on instruction and teachers while reducing 
spending on school administration or other support services. 

The first objective of this paper is to assess the causal fiscal impact of charter school expan- 
sion on traditional public schools in sending districts. By fiscal impact, we mean how charter 
expansion impacts (i) districts’ average per-pupil expenditures on non-charter students, (11) the 
share of these expenditures devoted to fixed and variable costs, and (iii) the share devoted to 
support services, instruction, and salaries. 

Understanding the potential fiscal impact of charter expansion is fundamental for at least 
three reasons. First, per-pupil expenditure may be a determinant of student achievement (Jack- 
son et al., 2016). If charter expansion has a fiscal impact, this might also affect non-charter 
students’ achievement. In addition, spending on fixed costs (typically building maintenance 
or debt interest) might not generate student progress to the same extent as spending on more 
variable costs such as instruction, textbooks, or teachers’ salaries. Second, we provide the first 
causal evidence of the fiscal impact of charter expansion in a state that provides temporary 
aid for traditional schools losing students to charters. Understanding whether the fiscal impact 
of charter expansion depends on the availability of such temporary support is key for policy 
makers across the U.S. Several states, such as Illinois, New Hampshire, New York, and Penn- 
sylvania, have adopted temporary refund schemes. Finally, the question of charter expansion’s 
fiscal impact on districts and traditional public schools has recently been at the center of a fierce 
debate. In November 2016, a ballot question to expand the presence of charter school in Mas- 
sachusetts drew national attention (The New York Times, 2016). A majority voted against the 
charter cap expansion in what is perceived nationally as a landmark decision. Given that about 
half of American states regulate charter expansion by setting caps, we expect more frequent 
discussions on the potential fiscal effect of raising these caps in years to come. 


3 Data and Descriptive Statistics 


We use data from three sources. First, we use the Massachusetts Students Information Man- 
agement System (SIMS) for the 2002-2003 through 2014—2015 school years. These files con- 


tain information on all Massachusetts public school students’ race, sex, ethnicity, reduced- 
price lunch status, special education status, English Language Learner status, community of 
residence, and current school. We use students’ current school to identify charter school 
students and to measure the percentage of students enrolled in charter schools in each dis- 
trict. Then we use student identifiers to merge SIMS demographic data with test scores from 
the Massachusetts Comprehensive Assessment System (MCAS) database, covering the years 
2003-2014. MCAS is administered each spring, typically in grades 3-8 and 10. Its database 
contains raw scores for math and English language arts (ELA). We standardized scores by sub- 
ject, grade, and year to have mean zero and unit variance in the population of students attending 
Massachusetts public schools. 

Information on districts’ expenditures comes from the Annual Survey of School System 
Finances collected annually by the Census Bureau (U.S. Census Bureau, 2017). All school 
districts that provide elementary or secondary education are included in the annual survey. The 
data include revenue by source; expenditure by object (instruction, support service functions, 
salaries, and capital outlay); and information on indebtedness, cash, and investments. Impor- 
tantly, a separate section indicates districts’ payments to charter schools.!® We can therefore 
isolate funds spent on non-charter students. 

Using this dataset, we build five outcomes to examine the fiscal spillovers of charter ex- 
pansion: per non-charter student, districts’ (1) total expenditures, and their expenditures on (2) 
fixed costs, (3) instruction, (4) salaries, and (5) support services. For each outcome, we divide 
the district expenditure on non-charter students by the total number of public school non-charter 
students in elementary, secondary, and high schools in that district. Expenditures on fixed costs 
are the sum of expenditures on plant operation and maintenance, student transportation, and 
interest on school debt. Expenditures on instruction correspond to expenditures for interac- 
tions between teachers and students in the classroom as well as co-curricular activities. These 
interactions can be activities of not only teachers but also of instructional aides or assistants 
engaged in regular instruction, special education, and vocational education programs.!? Ex- 
penditures on salaries include the salaries and wages paid by the district for all staff. These 
are gross salaries without deduction of withholdings for income tax, employee contributions to 
Social Security, and retirement coverage. Finally, expenditures on support services encompass 
student support (such as counseling), teacher support (such as training or instruction supervi- 
sion), and school administration.2° We use district identifiers to match data on expenditures to 


'8Charter schools with charters held by non-governmental operators — which covers almost all charter schools 
in Massachusetts — are considered beyond the scope of Census Bureau government finance statistics. In these 
cases, school district payments to charter schools are included within school district expenditures, but the finances 
of the charter schools themselves are excluded from the statistics. 

!Spending on instruction includes salaries. 

0More specifically, expenditures on support services are the sum of several expense types. First are expendi- 
tures for administrative, guidance, health, and logistical support, including social work, student accounting, coun- 
seling, student appraisal, information, and placement services, as well as medical, dental, nursing, psychological, 
and speech services. Expenditures on support services also encompass expenditures for instruction supervision, 
curriculum development, instructional staff training, academic assessment, and media, library, and instruction- 


state administrative education data for the years 2001-2002 to 2014-2015. 

To estimate charter effectiveness, we match the state administrative education data with 
admissions lotteries from 22 charter schools that enroll middle school students (in grades 5 
to 8) from the 2002-03 to 2013-14 school years. Appendix Table A.2 describes the charter 
schools that are eligible for the lottery instrument, as well as the years and grades of lottery. 
We exclude charter schools that closed, declined to participate, had insufficient records, were 
not oversubscribed, or served alternative students (like students at risk of dropping out). The 
resulting lottery sample includes 13 charter schools in Boston and 9 charter schools in other 
districts. 

Panel A of Table 1 reports statistics on all middle school students in Massachusetts in the 
2009-2010 school year, before the cap reform. In addition to being much more likely to be 
black or Hispanic and to receive subsidized lunch, students in high charter share districts have 
significantly lower math and ELA test scores than students in low charter share districts. These 
differences are even starker when comparing expanding and nonexpanding districts. 66.6% of 
students in expanding districts are black or Hispanic, as opposed to only 15.6% in nonexpand- 
ing districts. Students in expanding districts are also 49.5% more likely to have subsidized 
lunch, and they score 0.48 standard deviations lower in math and 0.59 lower in ELA. Such 
differences are not surprising given that charter schools generally open in disadvantaged areas 
and that the 2011 reform only raised the cap on charter schools for the districts in the lowest 
10th percentile of test scores. 

Interestingly, the differences shown in panel B, which reports districts’ average per-pupil 
expenditures in school year 2009-2010, are quite the opposite. High charter share districts 
spend on average $2,000 more per pupil than low charter share districts. Further, high charter 
share districts also spend more on instruction, fixed costs, support services, and salaries. 


4 Methodology 


One of the key difficulties in analyzing the impact of charter expansion is charter schools’ non- 
random initial location and expansion (Glomm et al., 2005; Bifulco and Buerger, 2015).”! To 
deal with this endogeneity, we exploit a 2011 cap reform in Massachusetts, which led to charter 
sector expansion. In the four years following the reform, the proportion of students attending a 
charter school jumped from 7% to 12% in the districts that expanded their charter sectors. The 
charter share remained relatively constant, at about 3%, in all other districts. For identification, 
we exploit this differential growth in expanding versus nonexpanding districts to generate an 
instrument for districts’ charter share. We simultaneously use a synthetic control approach to 


related technology services. Support tasks also relate to school administration, including expenditures for the 
principal and school office. 

*ICharter schools tend to locate in districts where the population is diverse, per-pupil expenditure is high, 
teacher costs are low, and public school achievement is relatively low (Glomm et al., 2005; Bifulco and Buerger, 
2015). 
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deal with potential non-random selection of districts into expansion. 

More formally, we identify an expanding district by looking at its charter sector growth 
before and after the 2011 reform. We examine how each district’s charter share changes be- 
tween 2002-2003 and 2014-2015. As displayed in Figure 1, we divide the full period into a 
pre-reform period that spans the years 2002-2003 to 2010-2011 and a post-reform period that 
spans the years 2011-2012 to 2014-2015. 7* We note as T'1 the first year of a period, TN the 
last year of a period, and N the number of years in the period. Then, for both the pre-reform 
and the post-reform periods, we calculate the slope of the charter share evolution (noted C’) as 
follows: (GaSe 

All districts for which the post-reform slope is larger than the pre-reform slope — that is, 
Slope post — Slope pyre > 0 — and that are in the lowest 10th percentile of student test scores 
are considered to be expanding districts. The following nine districts are expanding: Boston, 
Chelsea, Malden, New Bedford, Lynn, Gill-Montague, Lawrence, Winchendon, and Salem. 
Figure | shows a clearly accelerated charter expansion in these expanding districts after the 
2011 reform. We discard three districts from the group of expanding districts: Gateway, be- 
cause it experienced a decreasing charter share before the cap reform, and Lowell and Chicopee, 
because they only had very marginal changes in slopes after the reform and idiosyncratic evo- 
lution patterns that do not seem related to the cap reform. 

Low-performing districts that expanded their charter sectors after the reform might have 
different unobserved characteristics than low-performing districts that did not expand. Selec- 
tion into expansion might be driven by several elements. First, before the reform, the cap was 
not equally binding for all districts. Districts for whom the cap was binding might be expected 
to take more advantage of the reform to expand their charter sector. Second, low-performing 
districts can only increase charter enrollment if non-profit organizations, teachers, or parents 
decide to submit an application to either open a new charter school or expand an existing one. In 
addition, for districts that were close to the 9% limit on charter funding, only a proven provider 
may submit a new application. 

Another factor in district decisions about whether or not to expand its charter sector is 
tuition. Charter schools’ tuitions are determined by their sending districts’ per-pupil expendi- 
tures. Given the relatively large variation in per-pupil expenditures across districts, applicants 
for a new charter school might consider the per-pupil expenditures of the potential sending 
districts when deciding where to locate the new charter school.”? If, in addition, some dis- 
tricts faced increasing per-pupil expenditures before the reform, while others faced decreasing 
per-pupil expenditures, we might be concerned about selection on trends in unobserved char- 
acteristics. 

To address the selection into expansion among eligible districts, an obvious solution would 


?2We only look at expansions that are triggered by the 2011 reform. Charter schools have also opened and 
expanded in other years but to a significantly lesser extent. Between | and 5 charter schools opened in each year 
from 2002 to 2010, while 13 charter schools opened in 2011. 

3School districts’ revenues are largely based on property taxes. This creates large variations in revenues and 
per-pupil expenditures across districts. 
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be to use the eligibility criteria as instruments for the expansion. Unfortunately, most of the 
eligibility criteria are poor predictors of district charter expansion. The first stage is low. In- 
stead, we use the differential change in charter share in expanding and nonexpanding districts 
as an instrument. Using that differences-in-differences as an instrument raises the question 
of the appropriate control group of nonexpanding districts: Should we use all nonexpanding 
districts, low-performing nonexpanding districts that were eligible to expand, or a subset of 
these districts that are most similar to the expanding districts? To identify the appropriate 
control group, we start by building a synthetic control (SC) for the expanding districts. Be- 
cause SC has never been used in an instrumental variables framework, we develop a synthetic 
control instrumental variables approach (I[V-SC), which we complement with a more standard 
differences-in-differences instrumental variables (IV-DiD) estimator. 


4.1 Synthetic Control Instrumental Variables (IV-SC) 


We use the synthetic control strategy devised by Abadie et al. (2010) to construct a weighted 
average of control districts that matches the pre-charter-expansion outcome path in the expand- 
ing districts (Abadie and Gardeazabal, 2003; Abadie et al., 2010, 2015). Synthetic control 
outcomes in the post-reform period provide a plausible counterfactual for the treatment group. 
This allows us to estimate a reduced form treatment effect. One of our methodological con- 
tributions is to adapt the synthetic control methodology to estimate a first stage as well. Our 
objective is to find a group of control units with a pre-reform path that is similar to that of 
the expanding districts, not only in terms of outcomes but also in terms of charter share (the 
endogenous variable). In other words, we estimate both a reduced form and a first stage. 
More formally, let’s consider the following structural equation in which the charter share 
Cy¢ is the endogenous variable: 
Yje = V1 + PChe + vy (1) 


We want to estimate p, the effect of the charter share on our outcome of interest Y;,. We cannot 
estimate p from equation (1) directly by OLS because Cj; is potentially correlated with district- 
specific unobservables v;;. Therefore, we instrument C’;, with a dummy for expanding districts, 
£, that takes the value one for expanding districts and zero for the synthetic control districts. 
A dummy for expansion would clearly be endogenous when expanding districts are compared 
to all other districts. Comparing expanding districts to their synthetic control provides a more 


plausibly exogenous instrument. The first stage and reduced-form equations are: 


Cit = Y2 + BE je + Use (2) 
Yu = V1 aE ie cit (3) 


where a = (Gp. We use the following synthetic control procedure to estimate separately the 
reduced form treatment effect, a, and the first stage coefficient, 3. Consider a sample of J + 1 
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districts indexed by 7, and assume that district 7 = 1 will be the treated district (that is, the 
expanding district), while districts 7 = 2 to 7 = J+ 1 are potential control districts. The 
sample includes 7p pre-reform years as well as 7) post-reform years, with T’ = To + 7}. 

Yj(1) and Y;,(0) are the potential outcomes with and without treatment. The treatment 
effect for district 7 at time 7p can be defined as: 


aye = Yje(1) — Yje(0) = Vie — Yje(0) (4) 
We are interested in estimating the vector (@j,7 +1, -.., @;,r). This is the reduced form esti- 


mate of the IV-SC method. Abadie et al. (2010) show that we can identify the above treatment 
effects under the following model for the potential outcomes: 


Yje(1) = 6) Z 9; At by C1 jt € jt (6) 


Potential outcomes depend on a common factor 4;, a vector of observed covariates Z; that 
are not affected by the intervention, a vector of time-specific parameters 6;, a district-specific 
unobservable j1;, and an unknown common factor 2,;. €;; 18 a transitory shock with zero mean. 
Finally, aj; is a reduced-form year-specific treatment effect that is different from 0 only when 
j = 1andt > To. The model allows the impact of unobservable district heterogeneity to vary 
with time, unlike standard differences-in-differences or fixed-effect specifications that assume 
A, 18 constant over time. We can identify the first stage effect under the following model: 


Cee = Te Zp Oi RY EB St (7) 


The terms have the same interpretation as for the potential outcome. (3, is a first-stage 
year-specific treatment effect that is different from 0 only when 7 = 1 and t > To. 

Define a (J x 1) vector of weights W = (we, ...,w +41) such that w; > 0 and )>w; = 1. 
Each possible choice of W corresponds to a potential synthetic control for the treated district. 
The value of the outcome variable for each synthetic control (indexed by W) is: 


J+1 J+1 J+1 J+1 
So wiYie = b+ 0). wi Z; +d S° wyH; + So wyej (8) 
j=2 j=2 j=2 j=2 


The value of the endogenous variable for each synthetic control is: 


J+1 J+1 744 Ree 
So wiCin =mt oe) 9 wjiZj t+ me So wy + So wikis (9) 
i= j=2 j=2 j=2 
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Finally, assume a vector of weights (w5,...,w’j,,) that makes it possible to equalize three 
equations for each pre-reform year. First, the vector of weights equalizes the values of the 
pre-reform outcomes for the treated districts and the synthetic control. In addition, the vector 
of weights equalizes the values of the observed covariates Z; of the reduced form equation for 
the treated districts and the synthetic control. Formally, for each period t: 


J+1 J+1 
So wiVije = Yur and So wiZit = Zit (10) 
j=2 j=2 


Importantly, the vector of weights also equalizes the values of the pre-reform endogenous 
variable for the treated districts and the synthetic control: 


J+1 


SS) wiCje = Cy (11) 
j=2 


At that stage, it becomes clear that the pre-reform values of the endogenous variable should 
be identical (or as close as possible) in the treated districts and the synthetic control. To see 
why this is of first-order importance, imagine a situation where the vector of weights does not 
equalize the values of the endogenous variable for the treated and synthetic control. This could 
result in having a very good fit in outcomes between treated and synthetic control districts but a 
large difference in charter share. Given that charter share is a determinant of outcome, to obtain 
similar outcomes despite large differences in charter share, there must be other differences in 
unobserved predictors that compensate for the charter share gap. Finding weights that match 
not only the outcome variable but also the endogenous variable prevents differences in unob- 
served predictors between treated district and synthetic control districts. This is fundamental 
with respect to the independence assumption of the instrumental variables model. 


If the vector of weights (w3,...,w7,,) exists, Abadie et al. (2010) show that the reduced 


form treatment effect a; in equation 6 can be estimated by: *4 


J+1 
ofje = Yar — Dw HY (12) 
j=2 


4S ynthetic control weights are non-negative and sum-up to one. Doudchenko and Imbens (2016) propose 
a more general class of synthetic control estimators that allows researchers to relax some of the restrictions in 
the Abadie et al. (2010) method. They allow the weights to be negative, do not necessarily restrict the sum 
of the weights, and permit a permanent additive difference between the treated unit and the controls, similar to 
differences-in-differences procedures. 
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Using the same proof, we show that the first stage coefficient can be estimated by: 


7 J+1 
Bye = Cu — dS) wiCie (13) 


j=2 


The same weights are used for the first stage and the reduced form. This ensures that the 
control group is the same for both stages. 7° 

Considering a single treated unit and the effect of an intervention averaged over all post- 
intervention years allows us to omit the 7 and ¢ subscripts. The IV-SC estimator of the parameter 
p in the structural equation | is the ratio of the reduced form estimate © to the first stage B ye 


| &> 


PIv-sc = = (14) 


DR 


In practice, this [V-SC estimator can be obtained either by estimating the first stage and 
the reduced form separately, and then taking the ratio of the two, or by running a weighted 
two-stage least squares (2SLS) regression of the post-intervention outcome variable on the post- 
intervention instrumented endogenous variable. In this regression, each control unit is weighted 
based on the synthetic control weights, while the treated unit has a weight of one. When several 
units are treated, a synthetic control can be computed for each treated unit separately or for the 
group of treated units as a whole. We chose the latter option in our application. 

It should be noted that the synthetic control reduced-form estimate a’; is unbiased whereas 
the IV-SC estimator suffers from the standard bias of the 2SLS estimator (although it is con- 
sistent). This bias, however, might be limited. The 2SLS bias is an increasing function of 
the number of instruments. By definition, when only one unit is treated, the IV-SC estima- 
tor relies on a single instrument (a dummy for treated and synthetic control districts), and the 
just-identified 2SLS estimator is median-unbiased. 

Finally, note that although we call our method “IV-SC’”, it differs in important ways from 
methods that use instrumental variables to eliminate bias due to selection into treatment. We 
instead use the synthetic control method to estimate an unbiased reduced-form treatment effect, 
and we then scale this effect by the first stage, which we estimate using the same group of 
weighted control districts. We think, however, that the IV-SC terminology illustrates well the 
novel attempt to use a dummy variable for treated versus synthetic control as an instrumental 
variable. 


5In practice, using the same weights might yield a poor fit for the first stage or reduced form. In that case, 
using different weights might be more appropriate, although this would change the estimator. 

6Note that this IV-SC estimator could also be interpreted as simply providing an appropriate scaling for the 
reduced-form treatment effect of expansion, a. 


15 


Implementation 


In practice, let X, be the vector of pre-reform characteristics for the treated district (that 
is, the expanding districts) and Xg the matrix of the vectors of the untreated districts’ pre- 
reform characteristics. A novelty with the IV-SC is that X; and Xo should include the en- 
dogenous variable. The vector w* is then chosen to minimize the distance ||X; — Xow||V = 


J (X1 — Xow)/V(X1 — Xow) where V is a (k x k) symmetric and positive semidefinite matrix 
that represents the weight of each predictor variable. 

In practice, conditions 10 and 11 often only hold approximately. A perfect equality be- 
tween treated districts and synthetic districts can only be obtained if the values of the predictor 
variables for the treated units fall within the convex hull of the values for the potential synthetic 
control districts’ predictor variables. When the equality does not hold perfectly, it is standard 
practice to evaluate the discrepancy (or goodness of fit) by computing the root mean squared 
prediction error (RMSPE) as follows: 


1 TO J+1 2 
RMSPE = (Fi dM — wv (15) 


Predictor Variables 


We start by identifying outcome variable predictors, the most important of which is usually 
the lagged outcome variable because it accounts for the effects of any potentially unobserved 
predictor variables in pre-reform years. Including lagged outcome variable addresses con- 
cerns about omitting important unobserved predictors. Indeed, only units that are sufficiently 
similar in both observed and unobserved outcome variable determinants as well as in those de- 
terminants’ effects on the outcome variable should produce similar outcome trajectories over 
extended periods of time. We therefore include five years of lagged outcomes and five years of 
lagged endogenous variable (charter share) as predictor variables. Concretely, for each expendi- 
ture outcome, the predictors are the expenditure variables in the years 2003, 2005, 2007, 2009, 
and 2011. We use the same years for the charter share variable. For the main specification, 
we chose not to include additional predictor variables because the synthetic districts chosen in 
this manner are well matched to the expanding districts’ outcome. Appendix B presents sensi- 
tivity tests in which more and fewer lagged years are used as predictor variables and in which 
additional predictor variables are added. 


Donor Pool 


As a second step, we identify possible donor districts to create the synthetic control group. 
In order to avoid interpolation biases, it is very important to choose donor districts that have 
similar characteristics to the expanding districts. Specifically, since the expanding districts 
are all in the lowest 10th percentile of student test scores, we select similarly low-performing 
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districts as donor districts. We therefore restrict the donor pool to districts in the lowest 10th 
percentile when using districts’ expenditures as an outcome and districts in the lowest 25th 
percentile when using districts’ average test scores as an outcome. In addition, we drop districts 
that have an idiosyncratic shock to their endogenous variable (charter share). That condition 
is particularly important for [V-SC. Because the synthetic group is meant to reproduce the 
charter expansion that would have been observed for expanding districts without the 2011 cap 
reform, we discard from the donor pool two districts that experienced large increases in charter 
share after the reform, despite not being considered expanding districts, because their charter 
expansion did not accelerate after the reform.”’ 

We must strike a fine balance between improving the match quality by enlarging the donor 
pool and avoiding the overfitting that often results from including too many districts in the 
donor pool that are dissimilar to expanding districts. Overfitting is detected when expanding 
districts are matched to a large number of donor districts, many of which have a very small 
weight. In that case, there is a high risk that some synthetic districts are part of the linear com- 
bination despite having very different outcome determinants. Appendix B presents sensitivity 
tests in which we vary the size of the donor pool. 


Predictor Variable Weights 


Finally, the choice of the synthetic districts depends on the matrix of predictor variable weights 
V. We adopt a standard iterative optimization procedure that searches among all (diagonal) 
positive semidefinite V-matrices and sets of w-weights for the best-fitting convex combination 
of the control units. Best-fitting refers to the fit between the outcome of the treated districts and 
that of its synthetic control before the reform takes place (see Abadie et al. (2010)). Appendix 
B presents sensitivity tests in which we use the cross-validation method. 


4.2 Identifying Assumptions 


The synthetic control method assumes no interference between expanding and nonexpanding 
districts (Rosenbaum, 2007). In other words, increased charter attendance in expanding dis- 
tricts is assumed to have no effect on nonexpanding districts’ outcomes. This assumption is 
relatively plausible for fiscal spillovers. How much a district pays in charter tuition and how 
much aid it receives from the state depends exclusively on the number of students enrolled in 
charter schools in its zone. As a result, interference should be limited. Competition effects 
from charter school expansion are also likely to be limited at the district level. In expanding 
districts, 91% of charter school students come from the district in which the school is located. 


*7Formally, for each district in the donor pool, we calculate the post-reform charter expansion slope as previ- 
ously defined: (Crn—Cri) We also compute all expanding districts’ post-reform slope minimum. We discard 
two districts from the donor pool because they have larger post-reform slopes than the minimum slope of the 
expanding districts. 
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For surrounding districts, then, fear of losing students should be limited. In addition, the syn- 
thetic control method selects a limited number of control districts (usually between five and 
ten) out of a sample of 72 potential control districts. There is only a small probability that these 
synthetic districts are close enough to one of the expanding district to feel competitive pressure. 
Finally, even if expanding districts do have spillover effects on surrounding districts, this would 
most likely have an attenuating effect on our estimates. 

Standard instrumental variables assumptions apply when using the difference in charter 
growth between expanding districts and synthetic control as an instrument. First, conditional on 
pre-reform charter expansion trends, charter share should increase more in expanding districts 
than in nonexpanding districts. We show evidence of this first stage in Figures 3 and 7. 

Secondly, in our IV-SC context, the independence assumption states that mean potential 
outcomes are the same in expanding districts and in the synthetic control. As Abadie et al. 
(2010) show, this holds if treatment is independent of potential outcomes conditional on a set of 
unobserved time-varying common factors with time-invariant district-specific factor loadings. 
Note that our approach therefore allows for districts to select into charter expansion based on 
expected future trends in outcomes, so long as these trends have the factor variable structure 
described just above. In other words, future trends in potential outcomes can differ between 
expanding and synthetic control, so long as the post-reform trend difference has the same factor 
variable structure as the pre-reform trends. In practice, because our synthetic control matches 
well the evolution of the outcome variable in expanding districts during the ten years prior to 
expansion, observed and unobserved determinants of per-pupil expenditures are very likely to 
have evolved in the same way. Such a similar evolution before the cap reform makes it unlikely 
that charter providers expect per-pupil expenditures, as well as their observed and unobserved 
determinants, to evolve differently in the future in expanding and synthetic control districts. 

In our context, given the risk of non-parallel trends in district expenditures in expand- 
ing versus nonexpanding districts, allowing time-varying district unobserved confounders is 
key to attenuating endogeneity from omitted variable bias. From that perspective, the IV-SC 
method rests on identification assumptions that are weaker than [V-DiD estimators. While the 
differences-in-differences model controls only for confounding factors that share a common 
trend, the synthetic control method allows the effect of unobservable confounding factors to 
vary with time. 

Finally, the fact that districts reacted to the reform by expanding their charter sectors is 
assumed to only affect student outcomes through its effects on the probability of charter en- 
rollment, not through any other factor or unobserved characteristic. This exclusion restriction 
would hold if no other reform was adopted in 2011 or if the reforms adopted that year im- 
pacted student achievement equally in expanding and nonexpanding districts. Other reforms 
were indeed adopted in 2010: The Act Relative to the Achievement Gap included provisions 
for school turnarounds and the creation of innovation schools (Massachusetts State Legislature, 
2010). We show that our results are not driven by the introduction of these new schools in the 
robustness checks section. 
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4.3 Inference 


A caveat of the synthetic control method is that it does not allow us to assess the results’ sig- 
nificance using standard (large-sample) inferential techniques. There are two reasons for that. 
First, our analysis is not based on a sample of the population, but rather on the entire popu- 
lation of districts in Massachusetts. Due to the absence of sampling variation, all uncertainty 
surrounding the estimands is design-based. In other words, uncertainty in our context stems 
from unobservability of potential outcomes, in particular unobservability of what the treated 
districts’ potential outcome would be in the absence of the treatment (Abadie et al., 2017). 
Second, we only have a limited number of districts in the control pool and a limited number 
of post-reform periods covered by the sample. Typically, we usually have between five and ten 
districts in the synthetic control group and four post-reform years (from 2012 to 2015). These 
two reasons justify using permutation inference as described in Abadie et al. (2010). Following 
their approach, we sequentially apply the synthetic control algorithm to a random selection of 
nine districts drawn from the pool of potential controls.?8 Then, we compare the placebo effect 
with the treatment effect of the expanding district. Since none of the donor pool districts receive 
treatment, variation between each combination of nine placebo districts and its synthetic match 
occurs randomly. We can therefore assess the likelihood that the expanded districts’ measured 
treatment effect is due to chance and whether the treatment effect measured for the expanding 
districts is larger than that for districts chosen at random. 

We pay particular attention to the fit quality between each placebo district and its synthetic 
control. For each placebo district, the fit quality would only be good if its predictor variables’ 
values belong to the convex hull of these predictors’ values in its donor pool. In practice, the 
quality of the match might be poor if some placebo districts have very low (or high) achieve- 
ment or very low (or high) charter share. We therefore only consider placebo districts for 
which the root mean square prediction error (RMSPE) is not more than three times larger than 
the RMSPE of the expanding districts. We apply this rule for the RMSPE of both the charter 
share and the outcome (Ferman and Pinto, 2017).? 

Finally, we do inference separately for the first stage and the reduced form estimates. We 
cannot do placebo inference on the IV estimand because, by definition, the first stage estimates 
are close to zero for the placebo units. This implies that their IV estimand (the ratio reduced 
form over first stage) tends to infinity. 


8Our treated unit is an aggregate of nine expanding districts. To match the size of the treated unit, we also run 
the permutation inference on combinations of nine districts. 

>°Tn practice, there is no consensus on what pre-intervention fit is good enough for placebo units. Considering 
all placebo units might lead to over-rejection, while restricting the placebo units too much might lead to under- 
rejection (Ferman and Pinto, 2017). Considering placebo districts for which the RMSPE is not more than three 
times larger than the RMSPE of the expanding districts seems relatively conservative. 
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4.4 Differences-in-Differences Instrumental Variables (1V-DiD) 


As a second research design, we instrument the charter share using the interaction between a 
post-reform-years dummy and whether a student lives in an expanding district. We control for 
post-reform year and expanding district main effects. Our strategy therefore uses a difference- 
in-differences instrument. The reduced-form estimate of the social return is calculated as the 
change in achievement between the pre- and post-reform cohorts in charter expanding districts, 
minus the change in achievement in nonexpanding districts. 

The first-stage estimate corresponds to the charter sector growth differential between ex- 
panding and nonexpanding districts. Intuitively, students who applied to charter schools in 
expanding districts before the cap reform were significantly less likely to get a seat in a charter 
school than students who applied after the reform, when the number of seats had increased. 
This was not true in nonexpanding districts, where the number of seats in charter schools re- 
mained relatively constant. 

The IV-DiD rests on the idea that the pre-reform expanding districts’ cohorts provide a good 
counterfactual for what would have happened to post-reform expanding districts’ cohorts in the 
absence of the reform. Subtracting the changes in nonexpanding districts’ outcomes adjusts 
for any pre-post variation that affected expanding and nonexpanding districts equally over the 
period. This DiD successfully identifies charter school spillover under the standard parallel 
trends assumption that, absent the reform, the change in outcome over this period would have 
been the same in expanding and nonexpanding districts. We show graphical evidence of the 
first stage and parallel trends in our Results section below. 

Three additional identifying assumptions underlie the [V-DiD method. First, the interac- 
tion between living in an expanding district and post-reform years should be independent of 
potential outcomes. This would typically not hold if expanding and nonexpanding districts did 
not experience parallel pre-trends in expenditures or achievement before the reform. Due to 
concerns about such trends, we adopt a conservative approach by systematically controlling for 
districts’ time trends in our IV-DiD specifications. In addition, the interaction between living 
in an expanding district and post-reform years should only affect student outcomes through 
its effects on the probability of charter enrollment, not through any other factor or unobserved 
characteristic. We have discussed that exclusion restriction in the section on IV-SC.*° 

Finally, additional assumptions are required in our fuzzy-DiD environment (de Chaise- 
martin and D’Haultf(Euille, 2018). Both the expanding districts and the control group are 
always partially treated, in the sense that they have a non-zero charter share before and after 
the reform. This means we need two extra assumptions for identification in addition to the 
standard common trend assumption. First, in both expanding and non-expanding districts, the 
average treatment effect among districts that had a positive pre-reform charter share should 
remain stable over time. This assumption seems relatively plausible because the institutional 
features accompanying the charter expansion are the same before and after the 2011 reform. 


3°See Hudson et al. (2017) for a discussion of the assumptions underlying the IV-DiD. 
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In particular, the refund scheme does not change after the reform. The second assumption is 
that the treatment effect is the same in the treatment and in the control group. This assump- 
tion, however, is only required if the treatment intensity changes in the control group after the 
reform. Appendix figures A.2 and A.3 show that the charter share evolution for each of the con- 
trol groups we use is relatively stable post-reform. Any bias generated by differential treatment 
effects between the treatment and control groups should therefore be very limited. 

Using a DiD after the synthetic control method has two advantages. First, to construct a 
control group of nonexpanding districts, we start by re-using the group of districts identified 
and validated by the synthetic control method. We use these districts as a standard control 
group, without the district weights computed by the synthetic control method. Then, we enlarge 
this control group to include all nonexpanding districts in the lowest 10th percentile of the test 
scores distribution. In addition, combining IV-SC and IV-DiD gives us the opportunity to check 
that both methods yield similar results despite being based on different pre-trends assumptions. 

The second-stage equation for the spillover analysis is: 


Yar = Q2 + 62Pa + 02a + ACa + €at (16) 


where Y,, is per-pupil expenditure or achievement in district d in year t, 02 is the coefficient 
of the a post-reform dummy P,;, 92 is the coefficient of a dummy for expanding districts Ey, 
and €, iS an error term. Our treatment variable, C'z,, measures the share of students enrolled in 
charter schools. 

To instrument charter share, we combine a just-identified model and an over-identified 
model in which we use three instrumental variables. In the just-identified model, we use a 
dummy variable 74, as an instrumental variable for charter share. Z, is the interaction be- 
tween the post-reform dummy P,, and the dummy for expanding districts Eq. The first stage 
for this two-stage least squares (2SLS) procedure is: 


Cat = 01 + 01 Pit + 01. Fa + ¥Zae + Vat (17) 


where 7¥ is the effect of post-reform expanding districts on charter share. As in the second-stage 
equation, the first stage includes a post-reform dummy P,,, a dummy for expanding districts 
Eq, and district time effects. In the over-identified model, we use three dummy variables 7, ,;, 
Za, and Z3q as instrumental variables for charter share. The dummy variable for expanding 
districts is decomposed into three sub-categories: Boston, other urban expanding districts, and 
nonurban expanding districts. Z q 1s the interaction between a post-reform dummy and a 
Boston dummy. Z2,; is the interaction between a post-reform dummy and a dummy for other 
urban expanding districts. 734; is the interaction between a post-reform dummy and a dummy 
for nonurban expanding districts. In the over-identified model, the first stage for this two-stage 
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least squares (2SLS) procedure becomes: 


Cap = a + 6, Pap + 0, Ea + Zia + Y2Zoae + 3Zaae + Vig (18) 


For standard errors, we use the White estimator of variance. 


5 Results on Fiscal Spillovers of Charter Expansion 


Our results reveal that increased charter attendance both raises districts’ total per-pupil ex- 
penditures and shifts expenditures from support services to instruction. We use the synthetic 
control method separately for each of the five expenditure outcomes in which we are interested. 
For each expenditure variable, we use the log of the variable as our outcome. Table 2 lists the 
selected synthetic control districts and associated weights. Of the 75 districts in the donor pool, 
between five and 11 districts have been selected as synthetic control districts. For districts’ total 
per-pupil expenditure (column 1), Worcester is the most heavily weighted (44.3%), followed 
by Southbridge, Athol-Royalston, and Somerville, which receive weights of 19.4, 19.2, and 
14.9%, respectively. North Adams has the smallest weight at 2.3%. The next four columns 
report districts’ weights for per-pupil expenditures on fixed costs, instruction, salaries, and 
support services. 

The top left plot of Figure 3 shows the first stage estimate of the synthetic control method, 
that is, the charter share evolution in expanding districts and the synthetic control.7! Annual 
charter share in the synthetic group closely follows the charter share in expanding districts 
until 2011. The next five figures show the reduced form estimates, i.e., the evolution of our 
five measures of district expenditures in expanding districts and the synthetic control. Here 
again, the synthetic control appears to replicate very well the path of expanding districts’ per- 
pupil expenditures before the 2011 reform. The different synthetic groups appear to be good 
controls for the expanding districts. After 2011, however, the curves clearly diverge, and by 
2015, charter share in expanding districts has increased by more than three percentage points 
compared to the synthetic control, going from 5% to more than 8%.** Similarly, total per-pupil 
expenditures as well as per-pupil expenditures on fixed costs, instruction, and salaries increase 
in expanding districts. By 2015, total per-pupil expenditures have increased by, on average, 
4.8% in expanding districts compared to the synthetic control group; per-pupil expenditures on 
fixed costs have increased by 6.2%; per-pupil expenditures on instruction by 7.2%; and per- 


31We run the synthetic control algorithm separately for each outcome. As reported in Table 2, the group of 
synthetic control districts differs for each outcome, which also implies that we have a different first stage for each 
outcome. In practice, the first stage figures are relatively similar. Due to space restrictions, Figure 3 only shows 
the first stage for the per-pupil expenditure on instruction. 

?The magnitude of the charter share increase differs from the one in Figure 1 because in Figure 3, the charter 
share is an average of district-level charter share in expanding and nonexpanding districts. Figure | plots the 
charter share directly calculated from student-level data. 
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pupil expenditures on salaries by 5%. These increases are accompanied by a 4% reduction in 
per-pupil expenditures on support services.** 

We use placebo inference to evaluate the probability that these effects are due to chance. 
Figure 4 reports the estimated treatment effect for expanding districts (line “TREATMENT”’) 
and the placebo effect for each placebo district. We only keep the placebo districts that have 
a RMSPE that is no larger than three times the RMSPE of the expanding districts. We dis- 
card the control districts with high RMSPEs because they might bias the inference by creating 
spuriously large treatment effects. This explains why the number of placebo districts varies 
by outcome. The top left figure shows placebo inference for the first stage, in other words the 
comparison between the estimated changes in charter share for expanding districts and placebo 
districts. Expanding districts have very significantly higher charter expansion than any other 
group of placebo districts. The p-value, which calculates the probability of obtaining an es- 
timate at least as large as the one obtained for the expanding districts when the treatment is 
reassigned at random, is equal to zero. Changes in total per-pupil spending, spending on fixed 
costs, instruction, salaries, and support services also appear to be large, with p-values ranging 
from 0.051 to 0.162.*4 

The IV-DiD estimates confirm these results. When moving to the [V-DiD specifications, we 
successively use as a control group the synthetic control districts and the districts in the lowest 
10th percentile of test scores that did not expand. When we use the synthetic control districts, 
unlike in the IV-SC method, we do not use the districts’ weights. All synthetic control districts 
have a weight of one in the [V-DiD. Figure 5 reports pre-trends in both charter share and district 
expenditures when the control group comprises synthetic control districts. All trends look very 
parallel, although trends in fixed costs are slightly noisier. Appendix Figure A.1 reports trends 
when we enlarge the control group to all nonexpanding districts in the lowest 10th percentile of 
test scores. Again, pre-trends look very similar, except for trends in per-pupil expenditures on 
fixed costs. This confirms the importance of controlling for districts’ time trends in all ['V-DiD 
specifications. 

Table 3 reports two-stage least squares (2SLS) estimates of charter school expansion’s fiscal 
spillovers for the over-identified model. The coefficients suggest that overall per-pupil expen- 
ditures increases, although not significantly. However, as expected, per-pupil expenditure on 
fixed costs goes up. Moving from 7% to 12% of students attending charter schools (which is 
the average jump for expanding districts four years after the reform) would increase per-pupil 
expenditures on fixed costs by 5.8% per year. At the same time, per-pupil expenditures on 
instruction would also increase by 2.9% per year, meaning that increased per-pupil expendi- 
tures are not solely driven by fixed costs. With regard to the competition effect, districts seem 


33The value of the coefficients is displayed in Figure 4. The reported coefficient corresponds to the mean dif- 
ference, over the years 2012 to 2015, of the log of per-pupil expenditures in expanding districts and in synthetic 
control districts. Taking the exponential of these coefficients gives us the percentage increase or percentage re- 
duction in the absolute value of the outcomes. The increasing trends in per-pupil expenditures between 2000 and 
2010 are partly due to declining enrollment trends during that period. 

34The standard significance level of 10 percent corresponds to a p-value larger than 0.9 or smaller than 0.1. 
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to shift resources away from support services. For a 1.25 percentage point increase in char- 
ter share per year, we find that the aforementioned positive effect on instruction expenditures 
would be compensated by an 3.2% drop in support services expenditures per year. Interestingly, 
these estimates are very similar to the ones we obtain with the IV-SC method. 

Appendix Table A.3 shows that the first-stage coefficients for each of the three instruments 
are significant. The first stage is significantly larger in Boston, where post-reform charter share 
goes up by 5 percentage points more than in nonexpanding districts. In other urban districts 
and in nonurban districts, charter share goes up by 1.6 and 1.4 percentage points, respectively. 
Finally, Appendix Table A.4 shows results for the just-identified model. Standard errors are 
significantly larger when using a single instrument, which makes it more difficult to detect 
significant effects. 

Our findings on the fiscal spillovers of charter expansion are particularly interesting as they 
stand in contrast to prior studies that all find charter expansion has a negative fiscal impact 
on district spending. The fiscal impact of charter schools surely depends on whether or not a 
state has a refund scheme. Massachusetts has one, but the states previously studied (such as 
Michigan, Ohio, and North Carolina) do not, which might explain the negative fiscal spillovers 
observed by, for example, Arsen and Ni (2012), Ladd and Singleton (2018), and Cook (2018).*° 
On the other hand, Bifulco and Reback (2014) present case studies of the financial adaptation of 
traditional public schools to enrollment declines in Albany and in Buffalo, New York, where an 
aid program similar to the one in Massachusetts exists.*° Their estimates, made under different 
scenarios, suggest charter expansion has a negative impact. However, Bifulco and Reback 
(2014) do not measure causal effects of charter expansion, which might explain why our results 
differ. 

Our results show that districts facing the largest charter expansion tend to reduce their 
per-pupil expenditures on support services while increasing their expenditures on instruction. 
A competition effect might justify such a reallocation. When faced with the threat of losing 
students, traditional public schools might think more carefully about how to spend their limited 
resources in order to retain existing students and attract new ones. Because parents factor 
school ratings information into student enrollment decisions (Hastings and Weinstein, 2008), 
charter competition creates incentives for traditional public schools to reallocate resources in a 
way that boosts student achievement (Hoxby, 2003). 

The fact that schools switch their resources from support services to instruction also sug- 
gests they may perceive spending on instruction as more directly related to student progress 


35Cook (2018) finds that charter competition not only reduces state and federal revenues for traditional public 
schools, but also revenues raised through property taxes by depressing district-level residential property values. 

©Bifulco and Reback (2014) explain that New York State provides districts with increasing charter school 
enrollments transitional aid meant to reduce fiscal impacts on the district. “The aid program reimburses the 
districts for a portion of their charter school payments that are attributable to recent increases in charter school 
enrollment. The award amounts are computed as 80 percent of the payments attributed to increased charter school 
enrollment during the last year, 60 percent of payments attributed to increases in charter school enrollments two 
years earlier, and 20 percent of the payments attributed to increases in charter school enrollments three years 
earlier.” This aid is very similar to that in place in Massachusetts. 
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than spending on support services. However, some evidence shows that cutting spending on 
items such as pupil support, which is included in support services, is detrimental to students’ 
attainment (Carrell and Hoekstra, 2014; Carrell and Carrell, 2006; Reback, 2010). We inves- 
tigated this by decomposing support services spending into its main components and running 
a synthetic control for each; these results are in Appendix A. We do find a negative effect on 
pupil support spending, suggesting that cuts in support services may not be costless, though 
this effect is not statistically significant. 

Finally, the increased per-pupil expenditures generated by charter expansion and the shift 
of resources we observe from support services to instruction and salaries raises questions about 
the impact of charter school expansion on student achievement. A large literature has linked 
per-pupil expenditures and student achievement (Jackson et al., 2016; Lafortune et al., 2018; 
Hyman, 2017; Card and Payne, 2002; Hoxby, 2001). Jackson et al. (2016) find that a 10% 
increase in per-pupil spending each year for all 12 years of public school leads to 0.27 more 
completed years of education. In Massachusetts, Guryan (2003) finds that a per-pupil expen- 
ditures increase of | standard deviation increases test scores by approximately 0.5 standard 
deviation. 


6 Education Spillover of Charter School Expansion 


To investigate the impact of charter school expansion on student achievement, we use the same 
IV-SC and IV-DiD methodologies detailed above. However, looking at student achievement as 
an outcome raises two additional challenges in terms of identification. 


6.1 Change in Student Selection and Charter Effectiveness When the 


Charter Sector Expands 


We are interested in how charter expansion affects the achievement of those students who are 
left behind in the traditional public sector. Unfortunately, we cannot do this simply by esti- 
mating the causal effect of charter expansion on average traditional sector test scores. This is 
because charter expansion, by definition, also changes who is left behind in traditional pub- 
lic schools. Unless new charter enrollees are sampled randomly (or at least orthogonally to 
ability) from the traditional public school population, there will be a clear selection bias: ex- 
pansion affects the baseline ability of the average public school student, and therefore average 
test scores. 

Ample evidence confirms that this concern is not just theoretical. Selection into charter 
schools is non-random, correlated with achievement, and changes as charter schools expand. 
Using data from National Alliance for Public Charter Schools, Epple et al. (2015) find that 
the proportion of students eligible for free or reduced-price lunch (FRL) in charter schools has 
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grown markedly over time, from roughly 30% in 2001 to 50% in 2010. Baude et al. (2014) use 
data from Texas and find that student selection into charter schools moved from being negative 
in 2001 in mathematics and reading to roughly neutral in 2011. Finally, looking at charter 
schools that expanded in Boston after the 2011 cap reform, Cohodes et al. (2016) observe that 
expanded charters attracted a more disadvantaged, lower-achieving population. As suggested 
by the authors, “this pattern may reflect the changes in recruitment practices required by the 
2010 Achievement Gap Act, which mandated that charter schools take steps to enroll higher- 
need students”. 

In our own data, selection in charter schools also appears to change with expansion. As 
shown in Figure 6, before the reform, expanding districts were already growing in terms of 
charter share, while nonexpanding districts experienced no growth. Furthermore, charter stu- 
dents’ characteristics changed in expanding districts compared to nonexpanding districts. Fig- 
ure 6 reports that the share of black charter school students diminishes with expansion, while 
the share of Asian charter students increases with expansion. 

The dynamic selection of students into charter schools implies that students in traditional 
public schools are also an increasingly selected sample as charter schools expand. Therefore, 
simply regressing test scores on charter share within the sample of public school students would 
introduce a selection bias term that is correlated with charter share, and we will not recover 
causal estimates of the spillover effect. 

Unlike most previous studies, we address this problem by using a two-endogenous-variable 
approach which estimates the causal impact of charter expansion on all students, while ac- 
counting for the direct treatment effect of charter enrollment on charter students. Specifically, 
our regressions include both district-wide charter share and individual charter enrollment as 
explanatory variables. The effect of charter share when controlling for individual enrollment 
can then be interpreted as the spillover effect of charter schools. 

Controlling for individual charter enrollment requires us to account for student selection 
into charter schools: a large body of literature suggests that charter applicants have different 
observed and unobserved characteristics than non-charter applicants. We therefore use a lottery 
instrument to recover consistent estimates of charter effectiveness.’ 

The second challenge for identification is the potential correlation between charter expan- 
sion and charter effectiveness. The charter schools that expand are likely to be the best ones. 
As detailed above, after Massachusetts’ 2011 reform, expansions in districts close to the 9% 
limit on charter funding were limited to proven providers, that is, existing charter schools or 
boards of governors with track records of high performance. Consistent with this, Cohodes et 


37On charter school effectiveness and the use of lotteries to identify it, see Hoxby and Murarka (2007), Angrist et 
al. (2010), Dobbie and Fryer (2011), Dobbie and Fryer (2016), Abdulkadiroglu et al. (2011), Angrist et al. (2013), 
Abdulkadiroglu et al. (2016), Carlson and Lavertu (2016), Angrist et al. (2016), and Cohodes et al. (2016).). 
The model that controls for individual charter enrollment has both individual and aggregate charter enrollment 
as endogenous instrumented variables. This is a typical peer-effect specification as in Acemoglu and Angrist 
(2000) and Angrist (2014). A standard identification assumption of this model is that the private return of charter 
attendance with the reform instrument is the same as the private return with the lottery instrument. 
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al. (2016) find that, despite attracting a more disadvantaged, lower-achieving population, post- 
expansion charter schools in Boston produced larger effects than other charter schools before 
the reform.*® Figure 6 also shows that charter students’ test scores in both math and ELA have 
increased over time in expanding districts. Although this is not a measure of charter schools’ 
value-added, this evolution suggests there may be a correlation between charter expansion and 
charter effectiveness. Without accounting for that correlation, our estimate of charter expan- 
sions’ spillover effect would capture both the spillover effect and the effect of increased quality 
when charter schools expand. We account for changing charter effectiveness by allowing the 
charter effect to be time-varying. As far as we know, this is the first paper to account for both 
changing student selection into charter schools and higher-performing charter schools’ selec- 
tion into expansion. 

A last concern for identification arises if charter schools locate in districts that have ex- 
perienced decreasing or increasing achievement trends (Imberman, 2011). This makes the 
complementarity between IV-DiD (that assumes parallel pre-trends in outcomes) and IV-SC 
(that imposes parallel pre-trends) more valuable. Combining the two methods gives us the op- 
portunity to check that they yield similar results despite being based on different pre-trends 
assumptions. 


6.2 IV-SC: From Student-Level to District-Level Achievement 


While spending outcomes vary at the district-by-year level, student achievement varies at the 
student level. Yet the synthetic control methodology requires district-level variables to compute 
the district weights. We therefore need to aggregate our outcomes at the district level. In order 
to both control for individual-level confounders (charter enrollment and demographics) and 
aggregate the dataset at the district-by-year level, we start by running the following regression: 


Yiat = @ + BoC; + BC; * Urb; + BoC; * P; +9' Xi + wae + Cat (19) 


where Yj is the test score of student 7 in district d and year t. C;, is a dummy for individual 
charter enrollment, C;; * Urb; is a dummy for enrollment in an urban charter school, and C; * 
P; is a dummy for enrollment in a charter school after the 2011 reform. X; is a vector of 
student demographic characteristics (sex, race, special education, limited English proficiency, 
subsidized lunch status, and a female-minority interaction term). ; is a full set of district- 
by-year fixed effects that capture the remaining variation in achievement we are interested in. 
More specifically, the district-by-year fixed effects estimate the district-by-year level variation 
in test scores, once we have accounted for charter effectiveness and students’ demographics. 
This is the outcome variable we use in our synthetic control analysis. 

When controlling for the charter effect, we allow this effect to vary along two dimensions: 
whether the charter school is located in an urban area and whether the charter effect is estimated 


38Baude et al. (2014) also find that charter school quality has improved over time in Texas. 
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before or after the 2011 reform.*? In the second step, we simply apply the synthetic control 
algorithm using jig, as our dependent variable rather than the average Y;,. The rest of the 
IV-SC methodology is identical, as explained in section 4. 

To estimate equation 19, we instrument individual charter enrollment, enrollment in an ur- 
ban charter school, and enrollment in a charter school after the reform. We use as instrumental 
variables a dummy indicating if a student wins a charter lottery, a dummy indicating if a stu- 
dent wins a lottery for an urban charter school, and a dummy indicating if a student wins a 
charter lottery after the 2011 reform. Table 4 reports first stage and second stage estimates of 
the private return to charter schools. Columns 1, 2, and 3 show first stage coefficients for each 
of the three instrumental variables. They all have a positive and significant impact on student 
probability to enroll in a charter school. Coefficients in math and ELA differ slightly because 
of differences in student samples. The second stage coefficients confirm that charter schools 
produce larger gains in urban areas than in nonurban areas. Charters also appear to be more 
effective after the cap reform than before. This is consistent with what Cohodes et al. (2016) 
find in Boston. 


6.3 IV-DiD on Student-Level Achievement 


Unlike the IV-SC that requires aggregate level outcomes, we run the IV-DiD regressions on 
student-level achievement. To make sure that the achievement variable we use is as similar 
as possible for the IV-SC and the IV-DiD, we use the same residualization process. In other 
words, we use as an outcome the student level achievement once accounted for individual-level 
confounders (charter enrollment and demographics). We run the following regression to get the 
residualized test scores: 


Yiat = a + BoC; + BC; * Urb; + BoC; * P; + y'Xi + Eat (20) 


The only difference between this equation and equation 19 is the absence of the district-by- 
year fixed effects 44 in this equation. All other terms are identical. For the IV-DiD, we use 
€iat aS the dependent variable rather than the average Y;4,. The second-stage equation for the 
achievement spillover analysis 1s: 


Eidt = 2 + 02Piat + 02 Eig + AC iat + Ciat (21) 
where €;q; is the residualized test score of student 2 in district d and year t, 62 is the coefficient 


on a post-reform dummy P;,;, 2 is the coefficient on a dummy for expanding districts F';4, and 
€;qt 1S an error term. Our treatment variable, C’;,, measures the share of students enrolled in a 


>°A large body of evidence suggests that urban charter schools generate large academic gains for lottery appli- 
cants (Hoxby and Murarka, 2007; Dobbie and Fryer, 2011; Abdulkadiroglu et al., 2011; Angrist et al., 2016). We 
tested other sources of heterogeneity (including gender) and only kept the variables that were significant sources 
of heterogeneity. 
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charter school. 

For the analysis of achievement spillover, we restrict the sample of expanding districts 
to Boston. We adopt that restriction because Boston is the district with the largest charter 
expansion after the reform and the expanding district with the highest number of students in 
our sample. When testing the over-identified model with three instruments, the first stage 
coefficients of two instruments were not significant. In urban expanding districts, excluding 
Boston and nonurban expanding districts, the charter share did not increase significantly more 
than in nonexpanding distircts. We therefore prefer to focus the analysis on Boston.“° 

We use a dummy variable Z;,, as an instrumental variable for charter share. Z;q; is the 
interaction between the post-reform dummy P;4; and a dummy for expanding districts £4. The 
first stage for this two-stage least squares (2SLS) procedure is: 


Ciat = 01 + 61 Piat + 91 Bia + YZiat + Viet (22) 


where 7¥ is the effect of post-reform expanding districts on charter share. As in the second-stage 
equation, the first stage includes a post-reform dummy P;,4;, a dummy for expanding districts 
FEjqa, and district time effects. We cluster standard errors at the individual and district levels. 


7 Results on Education Spillover of Charter Expansion 


Our results show that increased charter attendance positively impacts traditional public school 
achievement, although the effect is not always significant. The synthetic control method re- 
veals a small improvement in student achievement in both math and ELA, though none of the 
changes markedly differ from placebo tests. Figure 7 shows the evolution of the charter share 
and math and ELA test scores in expanding and synthetic expanding districts. The top figure 
shows that charter share increases more in expanding districts than in synthetic control districts. 
The additional two figures demonstrate that the synthetic control very effectively replicates ex- 
panding districts’ achievement path before the 2011 reform. After 2011, the curves diverge, 
and by 2015, achievement in expanding districts has increased by 0.028 standard deviations in 
math and by 0.019 standard deviations in ELA, as compared to the synthetic control.*! 
However, placebo inference shows that these test score gains are not statistically different 
from gains in placebo districts. Figure 8 reports the estimate of expanding districts’ treatment 
effect as well as the placebo effects.*” This figure also shows the p-value, which is the proba- 


“0The fact that the first stage coefficients in the over-identified model are significant for the spending outcomes 
but not for the achievement outcomes is likely due to the different aggregation levels. The first stage coefficients 
are estimated on district-level variables for the fiscal spillovers and on individual-level variables for the achieve- 
ment spillovers. 

“The value of the coefficients can be found in Figure 8. 

* As for expenditure outcomes, we only keep the placebo districts with RMSPEs that are no larger than three 
times the expanding districts’ RMSPE. This explains why the number of placebo districts varies by outcome. 
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bility that one of the placebo coefficients is higher than or equal to the estimated coefficient for 
the expanding districts. If p-values for the first stage all show that charter share in expanding 
districts increased notably more than in nonexpanding districts, the p-values also show that the 
corresponding test score gains are not statistically significant. We run the synthetic control al- 
gorithm separately for math and ELA test scores. As reported in Table 2, the group of synthetic 
control districts differs for each outcome. This explains why we also have a different first stage 
for math and ELA test scores. 

When moving to the I[V-DiD specifications, we successively use as a control group the 
synthetic control districts and the nonexpanding districts with test scores in the lowest 10th 
percentile. Figure 9 reports pre-trends in student achievement when the control group com- 
prises synthetic control districts (top two figures) and bottom 10th percentile districts (bottom 
two figures). Some pre-trends look more parallel than others, which confirms that it is critical 
to control for districts’ time trends in all [V-DiD specifications. 

Table 5 reports two-stage least squares (2SLS) estimates of charter school expansion’s effect 
on student achievement. In all regressions, we use the interaction between the post-reform 
cohort and living in Boston as an instrument. The IV-DiD estimates show that charter expansion 
had a positive and significant impact on student achievement. Moving from 10% to 15% of 
students attending charter schools, which is the average post-reform increase in middle school, 
would raise non-charter student achievement by 0.033 standard deviations in math and 0.023 
in ELA. These estimates are remarkably similar to those we obtain using the IV-SC method. 
Using the districts in the lowest 10th percentile as a control group yields much smaller (and 
insignificant) coefficients. This is likely due to the fact that the synthetic control districts more 
effectively reproduce expanding districts’ achievement pre-trends than do the districts in the 
bottom 10th percentile. 

Broadly speaking, our results accord with previous studies showing charter expansion has 
limited impact on traditional public school achievement. Evidence from New York City (Win- 
ters, 2012; Cordes, 2017) and Florida (Sass, 2006) suggests mildly positive and sometimes 
significant effects on achievement. In contrast, Bettinger (2005) finds that charter expansion 
has a negative and significant, but very small, effect in Michigan. However, this paper primarily 
focuses on elementary schools rather than middle schools, which may explain the discrepancy. 
Imberman (2011) also finds significant negative effects for elementary schools but an insignif- 
icant positive effect, of similar size to our own results, for middle and high schools.” 

It is also worth noting that the negative results in the literature tend to occur in settings with 
little funding available to compensate public schools when the charter sector expands. This 
was the case for Bettinger’s setting in Michigan and could explain why his findings differ from 
ours. The positive effects in New York, meanwhile, come from a context, similar to our own, 
of increasing per-pupil funding as charters expand (Cordes, 2017). 


“3Ror additional references on charter schools’ effect on non-charter students, see Hoxby (2003), Booker et al. 
(2008), Zimmer and Buddin (2009), Davis (2013), Jinnai (2014), Mehta (2017), Cremata and Raymond (2014), 
Zimmer et al. (2009), Sass (2006), Bifulco and Ladd (2006). 
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The somewhat limited effect of charter schools on student achievement might be slightly 
surprising given the increased per-pupil spending observed. Two mechanisms help reconcile 
these two effects. First, the evidence on the effects of school spending on academic outcomes 
is mixed.** Second, it often takes time before additional spending starts to have an effect on 
student achievement. Jackson et al. (2016) find that, for low-income children, a 10% increase 
in per-pupil spending each year for all 12 years of public school is associated with 0.46 addi- 
tional years of completed education. Using a similar identification, Lafortune et al. (2018) find 
clear changes in achievement trends following the adoption of school finance reforms. These 
changes cumulate over subsequent years, so that "ten years after a reform, relative achievement 
of students in low-income districts has risen by roughly 0.1 standard deviation". Our identifi- 
cation only allows us to measure outcomes up to four years after the 2011 reform. However, 
consistent with the two aforementioned studies, we show evidence of larger long-term effects 
in the mechanism section. 


8 Robustness Checks 


Sensitivity Tests for Synthetic Control Specifications 


Identifying a group of synthetic control districts is the result of three successive choices regard- 
ing (1) the predictor variables, (2) the method used to compute predictor variable weights, and 
(3) the districts included in the donor pool. To mitigate potential concerns about specification- 
searching and cherry-picking, we run six robustness checks that test our main results’ sensitivity 
to changes in each of these three choices (Ferman et al., 2017; Kaul et al., 2017). Results are 
presented in Appendix B. Table A.7 details the specification used for each robustness check. 
For comparison, the first row presents the baseline specification used throughout the paper.*° 
Each robustness check departs from the main specification and changes one element at a time. 
We compare the different specifications in terms of the number of synthetic control districts 
identified, quality of the pre-reform fit for outcome variables and charter share (as measured by 
the RMSPB), and the reduced form treatment effect estimate. 

The sensitivity tests reveal that, for predictor variables, reducing the number of lagged 
values of the outcome variable and charter share or including their entire pre-reform path sys- 


“4Early observational studies found additional funding had small or zero effects (Coleman et al., 1966; 
Hanushek, 2003). Numerous papers have also documented the relationship between student achievement and 
school spending by exploiting exogenous variation in per pupil school spending caused by school finance reforms 
(SFRs). Card and Payne (2002) find that court mandated SFRs reduce SAT score gaps between low- and high 
income students. However, Hoxby (2001) finds increased spending due to SFRs has mixed effects on high school 
dropout rates. Looking at individual states, Guryan (2003), Papke (2005), and Hyman (2017) find that reforms 
improved test scores and college attendance in low-income districts in Massachusetts and Michigan. 

“In our main specification, to identify the set of synthetic control districts, we used five lagged values of 
the outcome and charter share as predictor variables. We used an iterative optimization procedure to compute 
the predictor variable weights. For the set of donor pool districts, we use districts in the lowest 10th and 25th 
percentiles of student test scores for districts’ expenditures and districts’ test scores, respectively. 
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tematically yields a worse fit on outcomes. Tests on the size of the donor pool confirm that we 
must strike a balance between having a sufficiently large donor pool, in order to have enough 
donor districts similar to the expanding districts, and not having too large a donor pool, to 
avoid overfitting. Using cross-validation to construct predictor variable weights — i.e., splitting 
the pre-treatment sample and choosing weights to minimize out-of-sample fit — instead of our 
nested optimization procedure tends to produce larger RMSPEs, possibly because splitting the 
sample introduces more noise with our limited number of years. Most importantly, for most 
sensitivity tests we run, the 2011 reform’s reduced form effect is notably consistent across spec- 
ifications. This is particularly true for districts’ expenditures, with the exception of per-pupil 
expenditures on fixed costs. 


Innovation and Turnaround Schools 


In 2010, the law that raised the cap on charter schools also included provisions for school 
turnarounds and the creation of innovation schools (Massachusetts State Legislature, 2010). 
The Innovation Schools initiative provided educators and other stakeholders in Massachusetts 
with the opportunity to create new schools that operate with increased autonomy and flexibility 
in terms of curriculum, budget, staffing, and school schedule and calendar. While the inno- 
vation school model aimed to be cost-neutral for districts, one-time competitive grants were 
introduced to support the development of these schools. 

The second initiative adopted in 2011 was school turnarounds. To better target assistance 
to underperforming districts, the Massachusetts Department of Elementary and Secondary Ed- 
ucation introduced a new five-level district classification system. The state’s most struggling 
schools are designated level 4 or 5 based on an analysis of four-year trends in absolute achieve- 
ment, student growth, and improvement trends.*° Districts with one or more level 4 or 5 schools 
are required by state law to develop Turnaround Plans that support the accelerated improve- 
ment of student achievement within three years. Plans may include nominating a new leader, 
called a receiver (in level 5 districts only); coaching activities for teachers, administrators, and 
district leaders; and developing work teams and professional communities of practice. In ad- 
dition, newly identified level 4 schools are eligible to apply for federal funding through the 
Massachusetts School Redesign Grants program. 

The instrumental variables exclusion restriction would not be verified if the introduction 
of innovation and turnaround schools was unbalanced between expanding and nonexpanding 
districts. To address this, for all years after the 2011 reform, we collected data on innovation 
schools, recipients of innovation schools grants, and the amounts received. We have also col- 
lected data on level 4 and 5 schools, school redesign grant recipients, and amounts received. As 
a robustness check, we re-analyze achievement spillovers by controlling for innovation schools, 
level 4 and 5 schools, and recipients of each grant type. This ensures that post-reform achieve- 


4©By statute, the state can have no more than 4% of all public, non-charter schools identified as Level 4 and 
Level 5 at one time. No more than 2.5% of the total number of districts can be designated Level 5 at any one time. 
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ment differences between expanding and nonexpanding districts are not driven by differences 
in the prevalence of innovation or turnaround schools or differences in grants amounts. 

We re-analyze fiscal spillovers by accounting for differences in grants received by expand- 
ing and nonexpanding districts. We subtract each district’s grants for innovation schools or 
school redesigns from its total expenditures. For other sub-expenditures (on fixed costs, in- 
struction, support services, and salaries), we calculate what share of the total expenditure they 
represent, and we use that share to subtract the grant received. For instance, if spending on 
instruction represents 60% of a district’s total spending, we would subtract 60% of the received 
grants from instruction expenditures. 

Table 6 indicates that our results on charter expansion’s fiscal spillovers are not driven by 
innovation and school redesign grants. Similarly, the addition of controls for innovation and 
turnaround schools hardly changes estimates of charter expansion’s achievement spillovers. 


9 Exploring Mechanisms 


9.1 Long-run Effects and the Role of Massachusetts’ Temporary Refund 


Scheme 


The treatment effect of expansion that we measure includes both any competitive effect of 
charter schools on traditional public schools and the direct effect of a short-term increase in 
traditional public school funding due to Massachusetts’ refund scheme. Separating these effects 
is useful both from an external validity perspective (what would we expect in states without 
refunds?) and a policy perspective (should more states adopt these refunds’). 

To provide some evidence on these mechanisms, we analyze whether the spillover effects 
of expansion persist after districts’ temporary aid ends. The 2011 reform we use above was 
too recent to measure post-refund effects of expansion. Therefore, we instead exploit, in an 
event-study framework, the fact that many districts in our sample saw charter openings pre- 
2011 which led to large, persistent, and often sudden increases in the share of students attend- 
ing charter schools. We select large openings similarly to how we chose expanding districts 
previously, as ones where the district saw faster growth following the opening than before. 
Specifically, we select openings where the total increase in charter share in the seven years 
after opening was at least 1 percentage point higher than the increase in charter share from the 
beginning of our sample period (2002) to the year of opening.*7 

We then use an event study to estimate the impact of these large openings on districts’ 
current and future charter share and outcome variables. That is, we estimate the following 


47Note that an opening at t denotes opening in the school year beginning at t, while the outcome variables 
are measured for the school year ending in ¢, so an opening at ¢ should only affect the charter share from ¢ + 1 
onwards. 
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equation by OLS: 
k=7 


Yar = a+ S- On geht a OE Val ee 
k=—5 


where Yz, is either the charter share or a financial or achievement outcome variable, and Oy, 
denotes that the district had a large opening at time ¢. We control for district and time fixed 
effects (nq and @;) and district-specific linear time trends (yqt). 0, is the estimate of the effect 
of a charter opening / years after the event. Note that this specification estimates the effect of 
charter openings up to seven years after the opening, by which point there would be no further 
reimbursement due to the initial increase. 

The coefficients 6; cannot necessarily be interpreted as causal effects because, as explained 
in previous sections, the location and timing of charter openings are likely to be nonrandom.*® 
However, we have two reasons to believe the results from this event study might be a good guide 
to longer-run causal effects. Firstly, by including 5 leads of charter opening (k = —1,...,—5), 
we are able to investigate pre-trends in outcome variables, and perform a Granger test for the 
direction of causality. Our results show that although the pre-opening coefficient estimates 
are noisy, there is no clear pre-trend for most outcomes. Secondly, our event study estimates 
turn out to replicate the synthetic control estimates surprisingly well over the three- to four- 
year timespan. These two factors give us some confidence that the event study provides useful 
information on the longer-run effect of charter expansion. 

Figure 10 plots the estimated effect of an opening at ¢t on outcomes att +k fork = 
—5,...,7. The effect at t = 0 is normalized to zero. The district charter share rises by 
about 1.5 percentage points in the year after opening, about 0.5 points in the year after, and 
subsequently remains largely flat. This is important, as the reimbursement formula depends on 
the increase in the number of charter students in past years. Hence, the average treated district 
should be receiving little refund money by ¢ + 7. 

Over the short term, expanding districts see a large and significant post-opening rise in total 
per-pupil spending and instructional spending, corresponding well to our IV-SC and IV-DiD 
results. Also in line with previous results, the effect on fixed costs is largely insignificant, and 
there appears to be a small positive effect on salaries. Slightly more surprisingly, spending on 
support services appears to increase, which contradicts our previous results. We tend, however, 
to trust the causal IV-SC and IV-DiD estimates more. 

The long-term effects are directionally similar to the short-term effects, though always 
smaller. This suggests that the largely positive effects of charter expansion on districts tend 
to persist, but more modestly, or vanish past the end of the temporary refund. 

The effects on achievement are modestly positive in the short run, again corresponding well 
to our IV-SC and IV-DiD results, and largest over the four- to six-year time horizon. This fits 
with previous research suggesting that it takes several years for increased spending to impact 


48We prefer not to do a synthetic control for these charter openings, as there is no clear way to both adjust for 
the fact that different openings happen at different times and simultaneously aggregate outcomes across districts 
to reduce noise. 
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achievement (Jackson et al., 2016; Lafortune et al., 2018). 

This distinction between the short-term and long-term effects also partially reconciles our 
results with the negative fiscal spillover effect identified by previous studies, which use data 
from states where districts are not refunded for their charter tuition expenditures (Arsen and Ni, 
2012; Ladd and Singleton, 2018; Cook, 2018). This event study provides suggestive evidence 
that our main results are driven largely by the temporary injection of extra funding into the 
school system, rather than any competitive effect. The small long-run effect is also consistent 
with the idea that the refund effectively “cushions the blow” to districts from a sudden loss of 
students due to charter expansion. 


9.2 Impact of Charter Expansion on the Pupil-Teacher Ratio 


We find that the districts facing the largest charter expansion tend to increase their spending on 
instruction. This might be due to a decreased pupil-teacher ratio in traditional public schools. 
Indeed, when facing charter school competition, over the short and medium terms, traditional 
public schools might be losing students without dismissing teachers. We confirm this by com- 
paring the evolution of the teacher-student ratio in expanding and nonexpanding districts. To 
do so, we apply the same IV-SC methodology as we have used for the main analysis. 

Figure 12 shows our results. The districtwide ratio of students to teachers falls in expanding 
districts relative to the synthetic control post-reform. Hence class size is also likely to fall. This 
reduction in class size could be one mechanism for explaining the positive achievement effects 
we observe, including over the longer run. Numerous studies have demonstrated the positive 
effect of smaller class sizes on student achievement (Krueger, 1999; Angrist and Lavy, 1999; 
Hoxby, 2000; Urquiola, 2006; Fredriksson et al., 2013) * and on longer-term outcomes, such 
as the probability of taking the ACT and SAT exams or being enrolled in college (Krueger and 
Whitmore, 2001; Chetty et al., 2011). However, according to our placebo inference the effect 
on the pupil-teacher ratio is not statistically significant.~° 


10 Conclusion 


The charter sector has grown rapidly since its introduction in the early 1990’s. Yet growing con- 
cerns have emerged about charter schools’ potential negative impact on non-charter students. 


“Hoxby (2000) finds no impact on U.S. data. 

Teacher transition from traditional public schools to charter schools is another potential mechanism through 
which charter school expansion might affect non-charter school students achievement. However, some evidence 
suggests that this mechanism is unlikely. Cohodes et al. (2016) examine the composition of Boston charter schools 
before and after the 2011 expansion. They show that in Boston 66 percent of the expansion charter teachers have 
less than one year of experience teaching in Massachusetts public schools and 25 percent of the expansion charter 
teachers came from the proven provider parent campus. This confirms that teacher transitions from traditional 
public schools to charter schools are very rare, in part because salaries are higher in traditional public schools than 
in charters. 
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Concerns that the charter sector drains resources and high-achieving peers from non-charter 
schools prevented charter expansion in Massachusetts in November 2016, when voters rejected 
a ballot initiative that would have added up to 12 new charter schools. This paper investigates 
the fiscal and educational impact of charter expansion on school districts by exploiting a 2011 
reform that raised the cap on charter schools in Massachusetts. 

Our results reveal that increased charter attendance increases per-pupil expenditures in tra- 
ditional public schools. In addition, these schools react to competition by shifting their re- 
sources from support services to instruction. The IV-SC method shows that, after the reform, 
total per-pupil expenditures, per-pupil expenditures on instruction, and salaries increased by 
5.2%, 7.2%, and 5%, respectively, in expanding districts compared to nonexpanding districts. 
This is accompanied by a 4% reduction in per-pupil expenditures on support services. Further, 
our results indicate that charter expansion positively impacts student achievement, although the 
effects are small and not always significant. Our estimates suggest that moving from 10% to 
15% of middle school students attending charter schools would increase non-charter student 
achievement by 0.033 standard deviations in math and by 0.023 in ELA. These results paral- 
lel previous studies that found charter expansion had limited impact on student achievement 
(Bettinger, 2005; Imberman, 2011). 

It is worth noting one additional caveat to our analysis. If charter schools have any spillover 
effect on student achievement in traditional public schools, we might expect the impact to 
be larger in traditional public schools that are geographically close to a charter school. As 
highlighted by Cordes (2017), examining spillover effects over large distances, as we do in 
this analysis, might underestimate the impact of charter schools on the performance of those 
students attending traditional public schools in the same neighborhoods where charter schools 
locate. Similarly, our estimates hold for districts where the share of students that attend a 
charter school raises from 5 to 12%. If the fiscal and education spillover effects are nonlinear, 
we would not recommend using our estimates to predict effects of charter expansions in vastly 
different ranges. 

Finally, our analysis focuses on the spillover effects of charter schools on traditional public 
schools. From a policy perspective, it is important to consider the effect of charter expansion 
on the entire school system. One might worry that if a temporary aid scheme for traditional 
school districts is necessary, charter expansion will be very costly to the state. However, this 
cost is partially compensated for by the reduced funding received by charter schools. Wolf et 
al. (2017) report that in Boston, charter schools’ per-pupil revenue is about 17% lower than in 
traditional public schools. 

Equally policy-relevant is the overall effect of charter expansion on student achievement. 
We find charter expansion has a small positive effect on the achievement of students who stay in 
traditional public schools. In addition, several papers show that urban charter schools in Mas- 
sachusetts significantly boost their students’ test scores, while nonurban charter schools seem to 
reduce student achievement (Abdulkadiroglu et al., 2011; Angrist et al., 2013; Abdulkadiroglu 
et al., 2016). Taken together, these analyses show that a student’s transition from a traditional 


36 


public school to a urban charter school can potentially have a large positive effect not only on 
her achievement but also on the achievement of students who stay in traditional public schools. 
The overall effect is more uncertain when students transition to nonurban charter schools. 
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Figure 1: Charter Sector Expansion after the 2011 Reform 
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Figure 2: Charter Sector Expansion after the 2011 Reform - Middle Schools 
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Notes: These figures plot the share of students attending a charter school over time. Figure 1 plots the share for 
elementary, secondary, and high school students, while Figure 2 is limited to middle school students. The plain 
lines represent districts that saw an increase in the share of students attending a charter school after the 2011 
reform (expanding districts), and the dotted lines represent the districts that did not expand their charter sector 


after the reform (nonexpanding districts). 
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Table 1: Descriptive Statistics for Students and Districts 


High Low Non- 
All charter-share _ charter-share Expanding expanding 
districts districts districts districts districts 
(1) (2) (3) (4) (5) 
A. Students’ characteristics 

Female 0.492 0.492 0.491 0.491 0.492 
Black 0.080 0.131 0.030 0.246 0.057 
Hispanic 0.138 0.227 0.052 0.420 0.099 
Asian 0.051 0.053 0.048 0.068 0.048 
Subsidized lunch 0.337 0.496 0.183 0.772 0.277 
Special education 0.176 0.188 0.164 0.198 0.173 
Limited English proficient 0.049 0.086 0.013 0.154 0.034 
Math test score 0.030 -0.162 0.216 -0.393 0.088 
ELA test score 0.024 -0.190 0.230 -0.496 0.095 

B. Districts’ per-pupil expenditures 
Total spending 14,402 15,614 13,573 15,817 14,357 
Spending on instruction 9,075 9,857 8,534 9,848 9,050 
Spending on fixed costs 2,275 2,363 2,207 2,264 2,275 
Spending on support services 2,998 3,263 2,827 3,421 2,985 
Spending on salaries 8,353 8,724 8,086 8,651 8,344 
Number of students 277,769 136,414 141,355 33,502 244,267 
Enrollment (share) 1 0.49 0.51 0.12 0.88 
Number of schools 1,185 625 601 204 1,009 
Number of districts 293 113 180 9 284 


1 Notes: The upper part of this table describes Massachusetts 5th-8th graders in 2009-2010, the year before the 
cap reform. The bottom part of the table reports districts’ per-pupil expenditures. In columns 2 and 3, districts’ 
charter shares are respectively higher (column 2) and lower (column 3) than the median value. Columns 4 and 5 
are restricted to districts where the charter sector expanded (column 4) and districts where the charter sector did 
not expand (column 5) after the 2011 reform. Statistics include Massachusetts middle school students for whom 
we have baseline characteristics. The lower part of the table describes districts’ expenditures for primary schools, 
secondary schools, and high schools. 
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Table 2: Synthetic Control Districts’ Weights 


Districts’ per-pupil expenditures Students’ test scores 
Fixed Support 

Total costs Instruction Salaries Services Math ELA 
District (1) (2) (3) (4) (5) (6) (7) 
Brockton 0.034 
Cambridge 0.167 
Chicopee 0.225 
Easthampton 0.044 
Erving 0.079 0.039 
Everett 0.116 
Fall River 0.161 0.122 0.138 
Greenfield 0.069 0.092 0.127 
Leominster 0.002 
Medford 0.209 0.006 
North Adams 0.023 0.088 0.076 0.024 0.175 0.179 
Northampton 0.213 
Oxford 0.075 
Randolph 0.075 
Somerville 0.149 0.100 0.235 0.234 0.084 
Southbridge 0.194 0.012 0.258 
Springfield 0.111 0.081 0.060 0.413 
Webster 0.357 
Winthrop 0.253 
Worcester 0.443 0.011 0.521 0.153 0.203 0.104 
Adams-Cheshire 0.053 
Athol-Royalston 0.192 0.154 0.045 
Pioneer Valley 0.022 


T Notes: This table reports the district weights assigned by the synthetic control method. Columns | to 7 report 
weights computed when the outcome variable is, respectively, districts’ per-pupil expenditures (column 1); 
districts’ per-pupil expenditures on fixed costs (column 2), instruction (column 3), salaries (column 4), and 
support services (column 5); and student achievement in math (column 6) and ELA (column 7). For all 
expenditure variables, we use the log of the variable as an outcome variable. 
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Figure 3: Charter Share and Districts’ Per-Pupil Expenditures in Expanding Districts and 
Synthetic Control Districts 


(a) Charter share (b) Total spending 
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Notes: This figure plots the share of students attending a charter school (plot a); districts’ per-pupil expenditures 
(plot b); and their per-pupil expenditures on fixed costs (plot c), instruction (plot d), salaries (plot e), and support 
services (plot f). For all expenditure variables, we use the log of the variable. The plain lines represent districts 
that saw an increase in the share of students attending a charter school after the 2011 reform (expanding districts), 
and the dotted lines represent the synthetic control districts. For expanding districts, we plot the average charter 
share and expenditures. For synthetic control districts, we plot the weighted average of the charter share and 
expenditures. We use the weights defined by the synthetic control method. 
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Figure 4: Placebo Inference for the Fiscal Impact of Charter School Expansion 
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Notes: This figure plots the distribution of the charter expansion’s effect on districts’ per-pupil expenditures, as 


measured by the synthetic control method. The lines "TREATMENT" report the coefficients when expanding 
districts are compared to their synthetic control districts. The exact value of each coefficient is reported in the top 


right corner of each figure. For all expenditure variables, we use the log of the variable. The other lines in the 


figures report the coefficients when a placebo group of non-expanding districts is compared to its identified group 


of synthetic control districts. The p-value is calculated as the probability of obtaining a placebo estimate that is 
greater than the actual estimated treatment effect (less than it when the effect is negative), multiplied by two to 
approximate a two-tailed test. 
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Figure 5: Pre-trends in Charter Share and Districts’ Per-Pupil Expenditures 
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Notes: This figure plots the share of students attending a charter school (plot a), districts’ per-pupil expenditures 
(plot b), their per-pupil expenditures on fixed costs (plot c), instruction (plot d), salaries (plot e), and support 
services (plot f). For all expenditure variables, we use the log of the variable. The plain lines represent expand- 
ing districts, and the dotted lines represent synthetic control districts. For both expanding and synthetic control 
districts, we plot the average charter share and expenditures without using the weights defined by the synthetic 
control method. 
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Table 3: 2SLS Estimates of Fiscal Spillovers 


Per-pupil expenditures on: 


Fixed Support 
Total Instruction costs services Salaries 
(1) (2) (3) (4) (5) 
Control group: Synthetic control districts 
Charter share 1.0980 2.3540** 4.6303 *** -2.6167** 0.9285 
(0.8633) (0.9485) (1.5172) (1.2673) (0.5827) 
N 196 196 182 224 252 
R2 0.999 0.999 0.968 0.988 0.999 
F-Stat 9.5 9.9 10.9 12.4 11.9 


Control group: Districts in the lowest 10th percentile 


Charter share 0.6311 1.6312* 4.3034*** -2.7500** 0.4498 
(0.8042) (0.8248) (1.4815) (1.1339) (0.5752) 
N 392 392 392 392 392 
R2 0.999 0.999 0.968 0.988 0.999 
F-Stat 10 10 10 10 10 


1 Notes: This table reports 2SLS estimates of the charter expansion’s effect on districts’ per-pupil expenditures. 
For all expenditure variables, we use the log of the variable. The endogenous variable is the charter share, 
which is a continuous variable that ranges from 0 to 1. In this over-identified model, we use three instruments: 
(i) the interaction between a post-reform years dummy and a Boston dummy, (ii) the interaction between 
a post-reform years dummy and a dummy for other urban expanding districts, and (iii) the interaction 
between a post-reform years dummy and a dummy for nonurban expanding districts. All regressions 
control for expanding districts, post-reform years, and district time trends. For standard errors, we use the 
White estimator of variance. When using the synthetic control districts as a control group, the number of 
observations varies for each outcome depending on how many synthetic control districts were identified. 

*** Significant at the | percent level. 
** Significant at the 5 percent level. 
* Significant at the 10 percent level. 
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Figure 6: Charter Students’ Characteristics and Achievement 
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Notes: This figure plots charter students’ characteristics. The plain lines represent districts that experienced an 
increase in the share of students attending a charter school after the 2011 reform (expanding districts), and the 
dotted lines represent all other districts that did not experience an increase in the share of students attending a 


charter school. 
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Table 4: Lottery Estimates of Charter Effects 


First stage 2SLS 
() (2) (3) (4) 
Math 
Charter 0.455*** -0.331** 
(0.0608) (0.117) 
Charter* Urban 0.312*** 0.932*** 
(0.0232) (0.126) 
Charter*Post Reform 0.497*** 0.0830** 
(0.0263) (0.0320) 
N 2985484 2985484 2985484 2985484 
F stat 400.53 398.88 318.77 
ELA 
Charter 0.456*** -0.160 
(0.0616) (0.0978) 
Charter* Urban 0.312*** 0.398 *** 
(0.0230) (0.106) 
Charter*Post Reform 0.495 *** 0.186*** 
(0.0267) (0.0262) 
N 2752583 2752583 2752583 2752583 
F stat 420.89 415.74 331.89 


1 Notes: This table reports first stage and 2SLS estimates of charter school attendance’s effects on student 
achievement. Columns 1, 2, and 3 show estimates of the first stage coefficients, and column 4 shows estimates 
of the 2SLS coefficients. There are three endogenous variables: a dummy for charter school attendance, the 
interaction between charter attendance and a dummy for urban schools, and the interaction between charter 
attendance and a dummy for post-reform years. We use three sets of instruments: a lottery offer dummy, a 
lottery offer for an urban charter dummy, and a lottery offer for a charter school after the reform dummy. Each 
endogenous variable is instrumented by the three instruments. However, for readability, we only report the 
coefficient of the relevant instrument in the first three columns, that is (1) the coefficient on the lottery offer 
dummy for the charter school attendance variable, (2) the coefficient on the urban charter lottery offer for the 
interaction between charter attendance and urban schools, and (3) the coefficient on the post-reform lottery 
offer for the interaction between charter attendance and post-reform years. All regressions control for race, 
sex, special education, limited English proficiency, subsidized lunch status, and a female by minority dummy. 
District-by-year dummies and risk set dummies are also included. Estimates pool post-lottery outcomes for 
grades 4-8 and cluster by student identifier as well as district. 

*** Significant at the | percent level. 

** Significant at the 5 percent level. 

* Significant at the 10 percent level. 


50 


Figure 7: Charter Share and Students’ Achievement in Expanding Districts and Synthetic 
Control Districts 
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Notes: This figure plots the share of students attending a charter school (plot a) and students’ average math and 
ELA test scores (plots b and c). The plain lines represent districts that saw an increased share of students attending 
a charter school after the 2011 reform (expanding districts), and the dotted lines represent the synthetic control 
districts. For expanding districts, we plot the average charter share (plot a), the average math test score (plot b), 
and the average ELA test score (plot c). For synthetic control districts, the plots represent the weighted average 
of the charter share or test score. We use the weights defined by the synthetic control method. The test scores 
used for this figure are the residuals of a regression of students’ raw test scores on a set of students’ demographic 
characteristics and a dummy for individual charter enrollment. 
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Figure 8: Placebo Inference for the Impact of Charter School Expansion on Student 
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Notes: This figure plots the distribution of charter expansion’s impact on student achievement, as measured by 
the synthetic control method. The lines "TREATMENT" report the coefficients when expanding districts are 
compared to their synthetic control districts. The exact value of each coefficient is reported in the top right corner 
of each figure. The other lines in the figures report the coefficients when a placebo group of non-expanding districts 
is compared to its identified group of synthetic control districts. The p-value is calculated as the probability of 
obtaining a placebo estimate greater than the actual estimated treatment effect (less than it when the effect is 


negative), multiplied by two to approximate a two-tailed test. 
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Figure 9: Pre-trends in Student Achievement 
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Notes: This figure plots student achievement in math (plots a and c) and ELA (plots b and d). The plain lines 
represent the district of Boston, and the dotted lines represent synthetic control districts. For the first two plots (a 
and b), when the synthetic control districts are used as the control group, the lines plot the average test score in 
these districts without using the weights defined by the synthetic control method. In the second panel (plots c and 
d), the control group is enlarged to all districts below the 10th percentile of student achievement. 
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Table 5: 2SLS Estimates of Charter School Expansion’s Impact on Achievement 


Math 
First Stage 2SLS First Stage 2SLS 
(1) (2) (3) (4) 
Control group: Synthetic Control districts 
Charter share 0.6639* 0.4654* 
(0.2648) (0.2234) 
Boston * Post-reform 0.0687*** 0.0659*** 
(0.0104) (0.0092) 
Boston 1.4789 0.2479 
(2.6863) (1.8233) 
Post-Reform -0.0206** -0.0175* 
(0.0104) (0.0092) 
N 316001 316001 338681 338681 
F-Stat 43.938 51.358 
R2 0.023 0.024 
Control group: Districts in the lowest 10th percentile 
Charter share 0.0647 -0.0492 
(0.2335) (0.3561) 
Boston * Post-reform 0.062 1*** 0.0621*** 
(0.0050) (0.0046) 
Boston -0.2847 -0.4500 
(1.8470) (1.7670) 
Post-Reform -0.0140*** -0.0137*** 
(0.0050) (0.0046) 
N 585920 585920 536720 536720 
F-Stat 151.647 183.208 
R2 0.045 0.039 


T Notes: This table reports first stage and 2SLS estimates of charter expansion’s effect on student achievement. 
The endogenous variable is the charter share, which is a continuous variable that ranges from 0 to 1. The in- 
strumental variable is the interaction between a post-reform dummy and a dummy for Boston. All regressions 
control for a Boston dummy, a post-reform dummy, and district time trends. Standard errors are clustered at 
the individual and district levels. 

*** Significant at the | percent level. 
** Significant at the 5 percent level. 
* Significant at the 10 percent level. 
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Table 6: 2SLS Estimates of Fiscal Spillovers 
Robustness Check Excluding Innovation and School Redesign Grants 


Per-pupil expenditures on: 


Fixed Support 
Total Instruction costs services Salaries 
() (2) (3) (4) (5) 
Control group: Synthetic control districts 
Charter share 1.0506 2.3060** 3.991 1 *** -2.6427** 0.9007 
(0.8579) (0.9404) (1.5103) (1.2675) (0.5864) 
N 196 196 182 224 252 
R2 0.896 0.851 0.685 0.746 0.866 
F-Stat 9.5 9.9 10.9 12.4 11.9 


Control group: Districts in the lowest 10th percentile 


Charter share 0.5783 1.5784* 4.2506*** -2.8028** 0.3970 
(0.8045) (0.8328) (1.4729) (1.1353) (0.58562) 
N 392 392 392 392 392 
R2 0.900 0.875 0.699 0.786 0.877 
F-Stat 10 10 10 10 10 


1 Notes: This table reports 2SLS estimates of the charter expansion’s effect on districts’ per-pupil expenditures. 
For all expenditure variables, we use the log of the variable, and we subtract the innovation and school 
redesign grants received. The endogenous variable is the charter share, which is a continuous variable that 
ranges from 0 to |. In this over-identified model, we use three instruments: (i) the interaction between a 
post-reform years dummy and a Boston dummy, (ii) the interaction between a post-reform years dummy and 
a dummy for other urban expanding districts, and (iii) the interaction between a post-reform years dummy 
and a dummy for nonurban expanding districts. All regressions control for expanding districts, post-reform 
years, and district time trends. For standard errors, we use the White estimator of variance. When using the 
synthetic control districts as a control group, the number of observations varies for each outcome depending 
on how many synthetic control districts were identified. 

*** Significant at the | percent level. 
** Significant at the 5 percent level. 
* Significant at the 10 percent level. 
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Figure 10: Event-Study Estimates of the Effect of Charter Opening 
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Notes: This figure plots the estimated coefficients on lags and leads of charter openings in our event study regres- 
sion. The dependent variable is either the charter share (plot a) or districts’ per-pupil expenditures (plot b to f). 
All regressions control for district and time fixed effects, as well as district-specific linear time trends. t + k in the 
graph above corresponds to the coefficient on the k‘” lag of a dummy for a charter opening. The solid line denotes 
estimates and the dashed lines 95% confidence intervals calculated from standard errors clustered at the district 
level. All regressions include district and time fixed effects and district-specific linear trends. District expenditure 
variables are in logs. 
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Figure 11: Event-Study Estimates of the Effect of Charter Openings 
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Notes: This figure plots the estimated coefficients on lags and leads of charter openings in our event study regres- 
sion. The dependent variable is districts’ average achievement in math (plot a) and ELA (plot b). All regressions 
control for district and time fixed effects, as well as district-specific linear time trends. t + k& in the graph above 
corresponds to the coefficient on the k*” lag of a dummy for a charter opening. The solid line denotes estimates 
and the dashed lines 95% confidence intervals (calculated from standard errors clustered at the district level). All 
regressions included district and time fixed effects and district-specific linear trends. Achievement is measured as 
MCAS scores standardized to have mean zero and variance | in each year. 
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Figure 12: Pupil-Teacher Ratio in Expanding Districts and Synthetic Control Districts 
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Notes: Figure (a) plots the log of the districtwide pupil-teacher ratio in the expanding districts and the synthetic 
control we construct for this variable. Figure (b) shows our placebo inference for figure (a). "TREATMENT" 
reports the average treatment effect measured in figure (a). The exact value of the coefficient is reported in the 
top right corner. The other lines in figure (b) report the coefficients when a randomly chosen placebo group of 
non-expanding districts is compared to its identified group of synthetic control districts. The p-value is calculated 
as the probability of obtaining a placebo estimate greater than the actual estimated treatment effect (less than it 
when the effect is negative), multiplied by two to approximate a two-tailed test. 
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Figure A.1: Pre-trends in Charter Share and Districts’ Per-Pupil Expenditures 


(a) Charter share (b) Total spending 
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Expanding districts ----- Nonexpanding districts | 


Expanding districts ----- Nonexpanding districts ] 


(c) Spending on fixed costs (d) Spending on instruction 
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(e) Spending on salaries (f) Spending on support services 
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Notes: This figure plots the share of students attending a charter school (plot a); districts’ per-pupil expenditures 
(plot b); and their per-pupil expenditures on fixed costs (plot c), instruction (plot d), salaries (plot e), and support 
services (plot f). For all expenditure variables, we use the log of the variable. The plain lines represent districts 
that saw an increase in charter school attendance after the 2011 reform (expanding districts), and the dotted lines 
represent districts in the lowest 10th percentile of student achievement that did not experience an increase in 
charter school attendance. 62 


Table A.3: First Stage Estimates of Fiscal Spillovers 


Control group: 


Synthetic control districts 


Per-pupil expenditures on: 


Fixed Support 
Total Instruction costs services 
(1) (2) (3) (4) 
Post-Reform * Boston 0.0497***  0.0502*** 0.0491***  0.0526*** 
(0.0095) (0.0095) (0.0092) (0.0093) 
Post-Reform * Other urban 0.0159*** 0.0164***  0.0153***  0.0188*** 
(0.0050) (0.0049) (0.0043) (0.0046) 
Post-Reform * Nonurban 0.0144** 0.0149*** 0.0138***  0.0174*** 
(0.0056) (0.0055) (0.0049) (0.0052) 
N 196 196 280 224 
F-Stat 9.5 9.9 10.9 12.4 


Salaries 


(5) 


0.0497*** 
(0.0091) 
0.0159*** 
(0.0041) 
0.0144*** 
(0.0048) 
252 
11.9 


Bottom 10 


districts 


(6) 


0.0469%*** 
(0.0091) 
0.013 1*** 
(0.0041) 
0.0117** 
(0.0048) 
392 
10 


t Notes: This table reports first stage estimates of charter expansion effects on districts’ per-pupil expenditures. 
For all expenditure variables, we use the log of the variable. The dependent variable is the charter share, 
which is a continuous variable that ranges from 0 to 1. In this over-identified model, we use three instruments: 
(i) the interaction between a post-reform years dummy and a Boston dummy, (ii) the interaction between a 
post-reform years dummy and a dummy for other urban expanding districts, and (iii) the interaction between 
a post-reform years dummy and a dummy for nonurban expanding districts. All regressions control for ex- 
panding districts, post-reform years, and district time trends. For standard errors, we use the White estimator 
of variance. When using the synthetic control districts as a control group, the first stage coefficients and the 
number of observations vary depending on how many synthetic control districts were identified for each out- 


come. 

*k* Significant at the | percent level. 
** Significant at the 5 percent level. 
* Significant at the 10 percent level. 
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Table A.4: 2SLS Estimates of Fiscal Spillovers - Just-identified model 


Per-pupil expenditures on: 


Total Instruction Fixed costs Support services Salaries 
(1) (2) (3) (4) (5) 
Control group: Synthetic control districts 
Charter share 1.9218 4.4483* 2.2171 -3.3946 0.7523 
(2.0438) (2.2917) (3.0962) (2.7929) (1.4828) 
N 196 196 280 224 252 
R2 0.873 0.820 0.687 0.694 0.830 
First stage F-Stat 15.2 16.6 19.7 24.3 22.8 


First stage coefficient 0.0191***  0.0196*** = 0.0186*** 0.022 1*** 0.0191 *** 
(0.0049) (0.0048) (0.0042) (0.0045) (0.0040) 


Control group: Districts in the lowest 10th percentile 


Charter share 0.4432 2.0078 2.8589 -3.8848 -0.5244 
(1.6718) (1.7837) (2.9886) (2.6931) (1.4644) 
N 392 392 392 392 392 
R2 0.872 0.849 0.700 0.720 0.829 
First stage F-Stat 16.6 16.6 16.6 16.6 16.6 


First stage coefficient 0.0164***  0.0164***  0.0164*** 0.0164*** 0.0164*** 
(0.0040) (0.0040) (0.0040) (0.0040) (0.0040) 


1 Notes: This table reports 2SLS estimates of charter expansion effects on district spending. The endogenous 
variable is the charter share, which is a continuous variable that ranges from 0 to 1. In this just-identified 
model, the instrument is the interaction between a post-reform years dummy and a dummy for expanding 
districts. All regressions control for expanding districts, post-reform years, and district time trends. For 
standard errors, we use the White estimator of variance. When using the synthetic control districts as a 
control group, the number of observations varies for each outcome depending on how many synthetic control 
districts were identified. The first stage coefficient is the coefficient of the interaction between a post-reform 
years dummy and a dummy for expanding districts. 

*** Significant at the | percent level. 
** Significant at the 5 percent level. 
* Significant at the 10 percent level. 
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Figure A.2: Charter Share in the Synthetic Control Group for each Per-Pupil Expenditure 
Variable 


(a) Total spending 
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—— Expanding districts —-——- Synthetic control group 
(b) Spending on fixed costs (c) Spending on instruction 
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> ae 
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Expanding districts —-——- Synthetic control group Expanding districts —-——- Synthetic control group 
(d) Spending on salaries (e) Spending on support services 
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34 
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Expanding districts —-——- Synthetic control group Expanding districts —-——- Synthetic control group 


Notes: This figure plots the share of students attending a charter school in the synthetic control group for each 
financial outcome variable, as a test of the “fuzzy-DiD’ assumption that the treated share in the control group 
remains constant. The graphs differ from one another only because different weights are used to construct the 
synthetic control for each outcome variable. The charter share in the treatment group is provided for comparison. 
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Figure A.3: Charter Share in the Synthetic Control Group for each Achievement Variable 


(a) Math (b) ELA 


15 


T T T T T T T T 
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Expanding districts —-——- Synthetic control group 


Expanding districts —-——- Synthetic control group 


Notes: This figure plots the share of students attending a charter school in the synthetic control group for each 
achievement outcome variable, as a test of the ‘fuzzy-DiD’ assumption that the treated share in the control group 
remains constant. The graphs differ from one another only because different weights are used to construct the 
synthetic control for each outcome variable. The charter share in the treatment group is provided for comparison. 
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Figure A.4: Placebo Inference for the Impact of Charter Expansion on Components of 
Per-Pupil Support Services Spending 
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Notes: This figure plots the distribution of charter expansion’s impact on components of support spending: pupil 
support (plot a), instructional support (plot b), general administration (plot c), and school administration (plot d). 
The “TREATMENT" lines report the coefficients when expanding districts are compared to their synthetic control 
districts. The exact value of each coefficient is reported in the top right corner of each figure. The other lines in 
the figures report the coefficients when a placebo group of non-expanding districts is compared to its identified 
group of synthetic control districts. The p-value is calculated as the probability of obtaining a placebo estimate 


approximate a two-tailed test. 


greater than the actual estimated treatment effect (less than it when the effect is negative), multiplied by two to 
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A Decomposing the Effect of Charter Expansion on Support 
Services 


Our results suggest that districts facing the largest charter expansion tend to reduce their per- 
pupil expenditures on support services while increasing their expenditures on instruction. Yet, 
evidence shows that cutting spending on support services is detrimental to students’ attainment, 
in particular when cuts target pupils’ support such as guidance, health and psychological sup- 
port, or social work (Carrell and Hoekstra, 2014; Carrell and Carrell, 2006; Reback, 2010).*! 

The results, reported in Figures A.5 and A.6, show that spending cuts in expanding districts 
mostly target pupil support and instructional staff support. Conversely, spending on general 
and school administration increases more in expanding districts than in nonexpanding districts. 
However, our permutation inference shows that (except for general administration) these es- 
timates are not statistically significant. Given that pupil support includes expenses related to 
counselors, cutting down this spending might negatively impact student achievement, unless 
other mechanisms compensate for this effect (such as class size reduction). 


5!Carrell and Hoekstra (2014) use within-school variation in the number of counselors in elementary school 
in Florida. They find that an additional counselor significantly increases achievement (particularly for boys), and 
reduces misbehavior by 20% for boys and 29% for girls. Using a similar identification, Carrell and Carrell (2006) 
find that lower student to counselor ratios reduce student disciplinary problems, in particular for minority and 
low-income students. Reback (2010) exploit discontinuities in Alabama’s elementary school counselor subsidies 
to show that additional counselor subsidies reduce the likelihood of disciplinary incidents (such as weapon-related 
incidents and student suspensions). They do not find any effect on student educational outcome however. To 
shed light on the effect that cutting down spending on support services could have on student achievement, we 
decompose overall support spending into its largest components: spending on pupil support, instructional staff 
support, general administration, and school administration.** 
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Figure A.5: Decomposition of Per-Pupil Expenditures on Support Services 


(a) Pupil support (b) Instructional staff support 
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Notes: This figure decomposes districts’ per-pupil expenditures on support services into expenditures on pupil 
support (plot a), instructional staff support (plot b), general administration (plot c), and school administration (plot 
d). For all expenditure variables, we use the log of the variable. The plain lines represent districts that experienced 
an increase in the share of students attending charter schools after the 2011 reform (expanding districts), and the 
dotted lines represent the synthetic control districts. For expanding districts, we plot the average charter share and 
expenditures. For synthetic control districts, we plot the weighted average of the charter share and expenditures. 
We use the weights defined by the synthetic control method. 
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2015 


Figure A.6: Placebo Inference for the Impact of Charter Expansion on Components of 
Per-Pupil Support Services Spending 
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(d) School administration 
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Notes: This figure plots the distribution of charter expansion’s impact on components of support spending: pupil 
support (plot a), instructional support (plot b), general administration (plot c), and school administration (plot d). 
The “TREATMENT" lines report the coefficients when expanding districts are compared to their synthetic control 
districts. The exact value of each coefficient is reported in the top right corner of each figure. The other lines in 
the figures report the coefficients when a placebo group of non-expanding districts is compared to its identified 
group of synthetic control districts. The p-value is calculated as the probability of obtaining a placebo estimate 
greater than the actual estimated treatment effect (less than it when the effect is negative), multiplied by two to 


approximate a two-tailed test. 


B_ Sensitivity Tests for the Synthetic Control Specifications 


Sensitivity tests for predictor variables 


For all results presented in the paper, we use five years of lagged outcome variables as predictor 
variables and five years of charter share. As explained in the methodological section, including 


Ta 


lagged outcome variables and lagged charter share is crucial to ensuring that these variables’ 
pre-reform trends are as similar as possible in expanding districts and the synthetic control. 
However, including too many lagged outcome variables might render other outcome predictors 
irrelevant (Kaul et al., 2017). In our case, the [V-SC method requires a good fit for both the 
outcome variable (for the reduced form estimates) and the charter share (for the first stage 
estimates). Therefore, it is important to check if the number of outcome and charter share 
lagged values impacts the fit quality for the reduced form and the first stage. In particular, we 
might worry that adding too many lagged values of the outcome variable makes it more difficult 
to get a good fit for the charter share. 

The sensitivity tests reveal that reducing the number of lagged values for the outcome vari- 
able and charter share from five to either three or one systematically yields a worse fit on out- 
comes, while fit on charter share is sometimes better and sometimes worse. For instance, when 
looking at districts’ total per-pupil expenditures, the pre-reform RMSPE of the reduced form 
goes up from 0.0179 with five lagged values to 0.0237 with three lagged values and 0.0248 with 
only one lagged value. Most importantly, reducing the number of lagged values often signifi- 
cantly increases the number of synthetic districts identified, a result that indicates overfitting. 
For instance, for districts’ total per-pupil expenditures, the number of synthetic districts jumps 
from five to 19 when the number of lagged outcomes shrinks. Including the entire pre-reform 
path of the outcome variable and charter share as predictors systematically worsens fit quality 
for districts’ expenditures. Such inclusion also generates a significantly worse fit for the char- 
ter share and a larger number of synthetic control districts when the outcome is districts’ test 
scores. 


Sensitivity tests for predictor variable weights 


We test two options for the method used to compute the predictor variable weights: a standard 
method and a cross-validation method. For the results presented throughout the paper, we em- 
ploy the standard method, which is an iterative optimization procedure that searches among all 
predictor weights matrices and sets of districts weights for the best-fitting convex combination 
of the control units. Best-fitting refers to the fit between the pre-reform outcomes of the treated 
districts and synthetic control. More specifically, the optimization problem uses four inputs: 
Xj , the vector of pre-reform characteristics for the treated district (that is, the expanding dis- 
tricts), Xo the matrix of the vectors of the untreated districts’ pre-reform characteristics, Y; the 
vector of pre-reform outcomes for the treated district, and Yo the matrix of the vectors of the 
untreated districts’ pre-reform outcomes. 
In the standard optimization method, the optimization proceeds in three steps: 


1. For given predictor weights V = (v1,...,vz), the donor weights w*(V) are chosen to 
minimize the distance ||X1— Xow||V = \/(X1 — Xow)/V(X1 — Xow) so that w*(V) > 
0 and the donor weights sum up to one. 


2. Given the donor weights w*(V), optimal predictor weights V* are chosen to minimize 
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|¥1 — Yow*(V)||? so that V* > 0 and the predictor weights sum up to one. 
3. The final donor weights are computed as w*(V*). 


The second method we test is the cross-validation method. This consists of dividing the 
pre-intervention period into two sub-periods: a training period and a validation period. The 
optimization follows the three steps presented above, with two exceptions: in step 1, the mini- 
mization only applies to the training period, while in step 2, the minimization only applies to 
the validation period. 

The results in Tables A.8 and A.9 show that using the cross-validation method produces 
lower-quality fit for outcome variables than the optimization method we use throughout the 
paper. Quality fit for charter share, however, is not always worse. 


Sensitivity tests for donor pools 


For most fiscal results presented in the paper, we use as a donor pool the districts in the lowest 
10th percentile of test scores. For achievement results and results on districts’ per-pupil ex- 
penditures on fixed costs, we use the districts in the lowest 25th percentile of test scores. For 
the latter outcomes that are more difficult to match, the larger donor pool markedly increases 
fit quality. That said, increasing the donor pool size also raises the risk of overfitting: a larger 
donor pool increases the chance that donor districts are matched because of idiosyncratic noise 
rather than an underlying trend shared with the expanding districts. Overfitting occurs when 
expanding districts are matched to a large number of donor districts, many of which have very 
small weights. Needless to say, we must strike a balance between having a sufficiently large 
donor pool, in order to have enough donor districts similar to the expanding districts, and not 
having too large a donor pool, to avoid overfitting. The results in tables A.8 and A.9 rule 
out concerns about overfitting for results on achievement and per-pupil expenditures on fixed 
costs. For these outcomes, the number of synthetic districts identified remains relatively con- 
stant when we increase the donor pool from the bottom 10th percentile districts to the bottom 
25th percentile districts. For all other outcomes, however, overfitting is systematic with the 
larger donor pool. When looking at districts’ average test scores for instance, more than 50 
districts are often identified as synthetic controls. 

Finally, and perhaps most importantly, for most sensitivity tests we run, the 2011 reform’s 
reduced form effect value is notably consistent across specifications. This is particularly true 
for districts’ expenditures, with the exception of per-pupil expenditures on fixed costs. For that 
outcome, as mentioned in the section on fiscal spillover, the reduced form estimate seems very 
sensitive to the specification adopted. 
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Table A.8: Results of Sensitivity Tests for the Synthetic Control Specifications 


Number of 
SC RMSPE of the RMSPE ofthe Treatment effect 
districts first stage reduced form (reduced form) 
(1) (2) (3) (4) 


Outcome: Districts’ Total Per-Pupil Expenditures 


Results reported in Figures 3 and 4 5 0.0026 0.0179 0.0487 
1 lag for outcome and charter share 19 0.0025 0.0248 0.0179 
3 lags for outcome and charter share 19 0.0012 0.0237 0.0477 
All lags for outcome and charter share 7 0.0028 0.0159 0.0500 
Additional predictor variables 7 0.0064 0.0163 0.0514 
Variable weights: Cross-validation 4 0.0070 0.0401 0.0179 
Donor pool: Bottom 25th percentile 55 0.0014 0.0162 0.0160 


Outcome: Districts’ Per-Pupil Expenditures on Fixed Costs 


Results reported in Figures 3 and 4 4 0.0174 0.0469 -0.0149 
1 lag for outcome and charter share 19 0.0022 0.0808 -0.0833 
3 lags for outcome and charter share 4 0.0103 0.0477 -0.0190 
All lags for outcome and charter share 4 0.0071 0.0428 -0.0388 
Additional predictor variables 5 0.0071 0.0450 -0.03 10 
Variable weights: Cross-validation 6 0.0020 0.1210 0.0299 
Donor pool: Bottom 25th percentile 11 0.0015 0.0379 0.0623 


Outcome: Districts’ Per-Pupil Expenditures on Instruction 


Results reported in Figures 3 and 4 5 0.0025 0.0229 0.0724 
1 lag for outcome and charter share 19 0.0019 0.0287 0.0520 
3 lags for outcome and charter share 19 0.0017 0.0306 0.0702 
All lags for outcome and charter share 6 0.0025 0.0151 0.0592 
Additional predictor variables 6 0.0058 0.0199 0.0678 
Variable weights: Cross-validation 4 0.0036 0.0326 0.0525 
Donor pool: Bottom 25th percentile 54 0.0013 0.0239 0.0125 


Outcome: Districts’ Per-Pupil Expenditures on Support Services 


Results reported in Figures 3 and 4 7 0.0024 0.0176 -0.0457 
1 lag for outcome and charter share 19 0.0028 0.0424 -0.0304 
3 lags for outcome and charter share 18 0.0054 0.0241 -0.0439 
All lags for outcome and charter share 8 0.0035 0.0162 -0.0424 
Additional predictor variables 7 0.0046 0.0190 -0.0348 
Variable weights: Cross-validation 5 0.0034 0.0532 -0.0497 
Donor pool: Bottom 25th percentile 54 0.0010 0.0128 -0.0346 


+ Notes: See next table. 
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Table A.9: Sensitivity Tests for the Synthetic Control Specifications (continued) 


Number of 
SC RMSPE of the RMSPE of the Treatment effect 
districts first stage reduced form (reduced form) 
(1) (2) (3) (4) 

Outcome: Districts’ Per-Pupil Expenditures on Salaries 
Results reported in Figures 3 and 4 9 0.0013 0.0177 0.0508 
1 lag for outcome and charter share 19 0.0020 0.0235 0.0303 
3 lags for outcome and charter share 19 0.0009 0.0220 0.0479 
All lags for outcome and charter share 8 0.0023 0.0110 0.0305 
Additional predictor variables 7 0.0072 0.0130 0.0372 
Variable weights: Cross-validation 9 0.0005 0.0229 0.0452 
Donor pool: Bottom 25th percentile 53 0.0014 0.0122 0.0264 

Outcome: Districts’ Average Test Scores in Math 

Results reported in Figures 7 and 8 7 0.0139 0.0090 0.0288 
1 lag for outcome and charter share 55 0.0446 0.1314 0.0496 
3 lags for outcome and charter share 270 0.0089 0.0211 -0.0088 
All lags for outcome and charter share 34 0.0543 0.0001 -0.0086 
Additional predictor variables 8 0.0253 0.0165 0.0440 
Variable weights: Cross-validation BS) 0.0094 0.0327 0.0284 
Donor pool: Bottom 10th percentile 7 0.0143 0.0090 0.0296 


Outcome: Districts’ Average Test Scores in ELA 


Results reported in Figures 7 and 8 6 0.0090 0.0150 0.0198 
1 lag for outcome and charter share 49 0.0245 0.0686 0.0075 
3 lags for outcome and charter share 228 0.0196 0.0143 0.0126 
All lags for outcome and charter share 49 0.0617 0.0003 0.0029 
Additional predictor variables 11 0.0118 0.0097 0.0289 
Variable weights: Cross-validation 6 0.0087 0.0475 -0.0515 
Donor pool: Bottom 10th percentile 6 0.0122 0.0118 -0.0112 


* Notes: This table reports results of sensitivity tests done for the synthetic control method. For purposes 
of comparison, the first row of each panel presents the baseline specification used throughout the paper. 
The upper panel shows results when the outcome variable is districts’ total per-pupil expenditures. Moving 
down the table, we document results when the outcome variable is districts’ per-pupil expenditures on 
fixed costs, instruction, and support services. The first column shows the number of synthetic control 
districts identified by the synthetic control algorithm. The root mean squared prediction error (RMSPE) 
measures the fit quality between expanding districts’ pre-reform path and nonexpanding districts’ outcome 
path (column 2) and charter share path (column 3). In column 4, the reduced form treatment effect estimate 
is the average post-reform gap between the expanding districts’ outcome and the weighted average outcome 
of the synthetic control districts. 
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